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FOREWORD 


THIS BOOK has the distinction of being the first of its kind, and a very signifi- 
cant first indeed. It marks the coming of age of a systematic conception of the 
application of psychological principles to the invention, development, and use 
of complex man-machine systems. As such it has many of the properties of 
a theory of psychotechnology—and a very broad-based theory, at that, when 
one considers the wide range of basic psychological knowledge that it focuses 
on the many issues that relate to the human component or components in a 
System. As a theory of the psychotechnology of man-machine systems, it 
achieves integration of what has heretofore been variously called *human 
engineering," “human factors engineering," or "engineering psychology" on 
the one hand and "personnel psychology" or "personnel and training research" 
on the other hand. This union comes easily and naturally once the concept of 
System is examined and once the full implications of the concept of the human 
being as a component of a man-machine system are recognized. For it imme- 
diately becomes clear that the available properties or functions of man must 
be considered in planning the mating of man and machine components to 
achieve the desired system function; that the desired properties and functions 
of man must be exactly specified (and sometimes revised) as system develop- 
ment progresses; that the desired characteristics of man must be achieved 
through selection and/or training techniques of high precision and efficiency; 
and that the functional efficiency of the human component must be maintained 
and tested within the system context not only on installation but also con- 
tinuously or at least periodically thereafter. 

The student or practitioner in engineering, psychology, personnel man- 
agement, or system design and development who reads this lucid, well- 
organized, and completely reasonable exposition of the interrelations and 
interdependencies of psychological principles in the design of systems may 
not realize the difficulties and misunderstandings that had somehow to be 
resolved before such an integrated statement could be presented. The psycho- 
technologies appropriate to personnel selection, training, and equipment design 
and use developed more or less in isolation from one another. In part this 
Was because different specialties and specialists within the science of psy- 
chology were involved in them, in part because the users of the technologies 
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placed the management of personnel matters, training, equipment design, and 
System design in more or less isolated compartments of the over-all manage- 
ment system. 

By far the most important factor producing resistance to the integration 
of these psychotechnologies has been the management system commonly 
employed by the military establishment and industry for the development and 
use of systems. During World War II. the technology of personnel selection 
and the technology of personnel training were insulated from one another in 
the Army Air Forces because these personnel subsystem functions were the 
responsibilities of different staff agencies. The schism was bridged only during 
the last year or so of the war by psychologists who knew that selection and 
training were but two aspects of a personnel management “package” or system 
and could not be properly specified in isolation. During this same time the 
"human engineers" concerned with equipment design werc moving in the 
direction of integration of equipment design and training considerations, but 
again with very limited success because they had the resistance of an inappro- 
priate management system. 

After World War II and before about 1953, only limited progress was 
made toward unification of the psychotechnologies. In the Air Force, the 
unification of personnel selection. and training, at least in research, was 
achieved in 1949 by the Air Training Command through the creation of the 
Human Resources Research Center (later to become the Air Force Personnel 
and Training Research Center), but the isolation of personnel and training 
technologies from the technology of human factors in equipment design per- 
sisted, since the latter was the management function of the Air Materiel 
Command (and later the Air Research and Development Command), 

Then, in the early 1950's, two events of great significance for the dev 
opment of the ideas in the present book took place. First, the Air F 
formalized the concept of the weapon system, its subsystems 
Secondly, it drew together within the Air Rescarch 
mand the research and development agencies conce 
equipment components, and their human components. While this still left the 
critical features of personnel management— selection, training, and mainte- 
nance of men—in managerial isolation from the research and development 
ns, a large net gain for the concept of a 
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technology of man-machine system design by the Air Force, Navy, and Army, 
and perforce by their contractors. Human-factors scientists and specialists 
are attached to design teams, system project offices, development groups, and 
system tests. Other specialists prepare detailed documents in which qualitative 
and quantitative predictions are made regarding the personnel required to 
operate and maintain systems under development, and they make statements 
about training and training-equipment requirements. And all this effort 
directed toward the human component of a future system explicitly recognizes 
the necessity of a “doctrine of concurrency"—that is, that the human com- 
ponents and the remainder of the system must be programmed to arrive at 
a man-machine assembly point at the same moment in time. Finally, much 
psychological research and development effort, both in-house and on contract, 
recognizes the obvious need for basic and applied research in support of this 
unification of the several originally disparate psychotechnologies in the con- 
text of man-machine system design, development, and utilization. 

Nevertheless, the idea that was born at least a decade ago, and that has 
grown steadily since then in spite of some temporary reversals, does not have 
the full acceptance and implementation that a casual examination of regula- 
tions, mission assignments, contract clauses, and research and development 
project statements might imply. In part this is because the management 
systems of the users of the technology have adapted only slightly to the idea 
of a personnel management system within which the unified technology can 
operate continuously on all aspects of the problem of the human component 
of complex systems. In part it is because only a portion of the human-factors 
scientists and specialists have had the time or opportunity to explore the 
integrated conception of psychological principles in system development. Thus 
it may be confidently asserted that this first presentation of this conception in 
book form will be the best of all possible kinds of milestones—it will serve 
as a synoptic and hence available and referenceable statement of the idea that 
has guided the most exciting and important advances in psychotechnology of 
the past decade; it will serve as the schema for the instruction of psychologists, 
engineers, and planners in the broad conception as well as the minutiae of 
this new psychotechnology; it will guide the definition of new basic and 
applied research and will assist in locating the points of impact of both old 
and new scientific knowledge; and if the managerial users of man-machine 
systems can be seduced into reading it, it may even participate in the correc- 
tion of management obstacles to the efficient employment of the unified tech- 
nology of man in man-machine systems. 

The editor of this important book, Professor Gagné, is eminently qualified 
to be the driving force and filter in this first effort to present an integrated 
psychotechnology of system development. No single person within psychology 
can claim to be the originator of this systematic conceptualization of psycho- 
technology; it is the product of many minds—of some basic scientists, some 


applied scientists, some professional military men. Among them are some who 
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may have contributed much to the notion but may also have delayed its 
realization by overemphasis on the importance, or ontogenetic priority, of 
one component technology or another. However that may be, all who know 
the intimate history of this idea over the last decade recognize Professor Gagné 
as one of the chief architects of the idea, one of the most persistent and critical 
originators of propositions and techniques for the transformation of the idea 
into procedural practice, and—as a consequence of his position as Technical 
Director of the Maintenance Personnel Laboratory of the Air Force Personnel 
and Training Research Center—one of the leaders in the actual implementa- 
tion of the idea in the case of airborne fire-control systems and the early 
ballistic missile weapon systems. His leadership is attested by the outstanding 
and widely representative authors of the chapters of this book. 


ARTHUR W. MELTON 
Ann Arbor, Michigan 


October 1961 
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PSYCHOLOGICAL PRINCIPLES 
IN SYSTEM DEVELOPMENT 


INTRODUCTION 
Robert M. Gagné 


In our society, some of the most important and spectacular developments 
of the current age are systems. Although the invention of individual machines 
is still taking place, we tend to think of this kind of invention as being charac- 
teristic of a previous “golden age,” often associated with what historians call 


the industrial revolution. In a general sense, machines were conceived as 
and manipulating the environment. 


extensions of man’s capacity for perceiving 
They were tools which helped him to see in the dark, to lift heavier loads, to 


Move objects around more rapidly, to perform delicate operations more pre- 
cisely, to record and retrieve quantities of words, pictures, and sounds, and 
to do many other things. But as time went on, it became apparent that the 
social potentialities of machines as tools was far overshadowed by the possi- 
bilities exhibited when machines of somewhat different design were embedded 
within a complex organization, or system. Thus the electric light soon came 
to be, not simply a tool (a lamp), but a complex "lighting system" which 
later evolved into a "power distribution system." The telephone became not 
simply a tool for transmitting and amplifying the human voice but a com- 
munication system. Such has been the trend of history for a great many inven- 
tions which began as machines. 

To progress from a simple machine like the telephone toa complex 
communication system required many additional inventions, including among 
Others such devices as central switchboards, methods of multiple transmission 
in single wires, and, in more modern times, the dial system. Such inventions 
had initially to be conceived, as well as designed and developed, within a total 
Conception of a system. A similar picture can be drawn of the development of 
Most military systems. The airplane was initially conceived as a tool which 
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enabled a man to overcome the force of gravity and propel himself and other 
bodies through the air. Fitted with another kind of tool, a machine gun, it 
was capable of directing fire at targets on the ground or at other aircraft in 
the air. But it was not many years before the requirements for increased 
speed, altitude, and maneuverability of the fighter aircraft forced the design 
and development of something quite different and more highly sophisticated— 
an interceptor aircraft system. The “tools” were, in effect, taken out of man's 
hands, and they as well as the man himself became only parts of a complex 
System of radar detection, communication, and control. In such a system, all 
of the parts had to be conceived and built to fit together and function as a 
total organized whole. 

Although it is evident that many systems in the past evolved by steps, 
nowadays it has become quite commonplace for designers to take from the 
very outset the deliberate course of deriving from some originally stated 
purposes the characteristics of a total organized system. With such a concep- 
tion, the various parts of the system can no longer be thought of as tools for 
the extension of man’s capacities but instead must be designed in such a way 
as to integrate their functions with other parts of the system in the accom- 
plishment of system purposes. As an example, an oscilloscope cannot appro- 
priately be conceived in a systems framework as merely a tool for displaying 
wave forms to man; rather it must be looked upon as a link in a communica- 
tion chain whose function is to transform one variety of information coding 
into another. The determining ideas in such a conception are the system's 
goals, in relation to which any part, whether man or machine, has a sub- 
ordinate, albeit essential, function. As a consequence, both kinds of system 
components—man and machine— must be planned from the very outset with 


Systems is not an activity which has been performed 
years, in an entirely systematic way. Rather, such s 


something which has evolved in quite recent years. Up until the early 1950s 
at the latest, the human 
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need for systematic planning of human components is based upon the desire 
to achieve maximal efficiency of system development. 


PLANNING FOR HUMAN COMPONENTS 
IN SYSTEM DEVELOPMENT 


[6] 


As our previous discussion implies, planning the development process 
for human components of a system must be thoroughly integrated with the 
planning for machine development. Of course one does not acquire human 
beings by constructing them, as one does machines. Nevertheless, the processes 
by means of which human components are "developed" is just as orderly, in 
its way, as is the process of machine development. Development pertaining 
to human components has its own techniques, procedures, and tools. It also 
takes time, and therefore must be integrated with the planning of system and 
subsystem from the very earliest stages, in order that delays be avoided or 


ineffective compromise be eliminated. 


The Design Stage 


System design begins with a statement of purposes for the system, one 
or more “missions” the system is expected to perform. The purposes set the 
stage for the derivation of what the system's characteristics will be. Before 
going any further, systematic plans must be made for how the system is to 
Work, and this means not only that the machines must be conceived func- 
tionally, but that there must also be a design for operations. More and more 
frequently in modern systems, the conduct of "advanced design" studies is 
Coming to be an enterprise of increasing importance and thoroughness, with 
the recognition that some of the most crucial decisions affecting the system's 
ultimate usefulness are made at this point. Furthermore, it is recognized that 
human capabilities and human functioning constitute essential factors in the 
Statement of planned operations which results from these decisions. "Opera- 
tions," after all, must be conceived as prospective events that human beings 
do with and to machines. ^ 

From these plans for operations, together with the knowledge that 
System designers possess of the current state of technological knowledge, are 
derived decisions about the functions of subsystems, the major parts of the 
total system, and the ways in which they may be connected together to fulfill 
the system goals. At this stage, too, some highly important judgments are 
Made with regard to human beings, and often some entirely crucial ones. 
Whether explicitly or not, designers at this stage are engaged in decisions that 
determine what kinds of Auman functions will be employed as well as what 
Machine functions are to be used. If costly mistakes are to be avoided, con- 
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Figure 0.1. The procedures used in the development of human com- 
Ponents of systems, and their order of initiation, in relation to the stages of 


System development. Procedures used for equipment development are not 
shown in detail. 
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parallel paths: machine components and human components. They are not 
independent, of course; rather, they interact at many points as development 
proceeds. But they use different techniques. Here we shall follow the line of 
human component development, and indicate how and at what points such 
interactions occur. 

Once the purpose and function of a subsystem has been stated, the 
designer of human components can proceed to describe in specific terms the 
nature of the human functions being utilized. This is the job of task descrip- 
tion, whose basic role in the unfolding process of development is easy to 
understand. The description of tasks provides the kind of information to 
which all subsequent plans for human beings in the system must constantly 
be referred. These are the statements which specify exactly what it is that 
the man-machine combinations comprising the subsystem are doing. Although 
begun at this early stage of system development, such descriptions, it is 
apparent, must be continually revised and refined, to reflect changes in 
machine design, throughout the development process. 

Initially, task descriptions make it possible for the psychological designer 
to perform the two activities that underlie all the rest of his work: to design 
jobs and to undertake the task analysis which makes possible decisions about 
the techniques to be used in achieving the human behavior required for these 
jobs. Job design may be considered to be primarily a matter of determining 
how individual tasks may best be put together (“packaged jp so that they 
can most efficiently be performed by single individuals. Consideration has to 
be given to the number of tasks, their length, and their physical location 
within the subsystem. In addition, the attempt is made to combine into single 
jobs tasks which require similar human capabilities, so that more or less 
specialization of human functioning will result. The analysis of tasks is under- 
taken to determine to what extent cach kind of human behavior required 
can be achieved with the use of the various techniques available: by providing 
job aids (job “supports”), by selection, by training. A 

At this point the designer of human components has considerable basic 
knowledge of what kinds of behavior are required within the system. He now 
knows what human beings will be required to do, and in a general way what 
procedures will need to be followed to insure that people can meet these 
requirements. It is now time for another set of decisions which make demands 
upon the technology derived from psychological science. Can the required 
behaviors be achieved by providing job aids to facilitate human performance? 
Can they be obtained by selection of people with the right kinds of funda- 
Mental abilities? To what extent must the capabilities needed be established by 
training? The result of these decisions may be a set of a pA s which 
Specify the goals of each of these sets of procedures. Obviously, es ecisions 
must be made together, since they mutually influence each other: if the human 
tasks can be simplified by job aids, less stringent selection and training may 
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have to be done; the more precise and specialized selection can be made, the 
less training will be needed; and so on. Just as obviously, these decisions, if 
they are to be wisely made, demand the most careful and thoroughgoing use 
of the facts and theories of the science of psychology. 


The Development Stage 


The actual development of the procedures needed to carry out these 
processes can now be undertaken, once their requirements have been estab- 
lished. Job aids can be developed to provide for storage of information beyond 
the capacity of the human memory, or to serve as external cues for the 
instigation of behavior required in system tasks. Most commonly, these addi- 
tions to the system take the form of check lists and instructional manuals. 
Aptitude tests can be developed or chosen from existing stocks to measure 
the basic abilities that have been identified, within programs of personnel 
selection and classification. And procedures can be designed for individual 
training, based upon psychological principles of learning relevant to the kinds 
of performances needed. 

A variety of purposes must be served by procedures of training. First of 
all is the need to establish skills which pertain to the performance of 
relation to a machne or to a set of tools; this is the 
training. But still a different kind of technique is nee 
effective functioning of various human interactions 
the subsystem or between subsystems. Men must no 
bilities of Operating machines, but also of communi 
ways which will bring 
under a wide variety of 
dures required to bring 
known as team training. 
beings acquire and refin 
techniques brings us to 
concept of system training. 


a man in 
province of individual 
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ally, system training is sometimes conceived as a part of the operation called 
the "system test," and may occur over a period of years during which a 
system attains a steadily increasing degree of operational efficiency. 

One of the particular techniques that has achieved considerable success 
in the conduct of training is the use of training devices. Such devices, of 
course, have an equipment development cycle of their own, which must be 
keyed in with the development of system equipment. Training devices have an 
important role to play, not only in establishing the specific skills of machine 
operation, but also, in the form of simulators, for the conduct of team training 
as well as system training. 

The purpose of training of any and all sorts is to bring about in human 
beings a capability of some kind of performance of which they were formerly 
incapable. This being the case, it is evident that a means must be provided to 
measure the results of training—to determine whether the desired capability 
has in fact been established. This circumstance calls for the development of 
performance measures, a general name for a set of techniques used to assess 
human performance. The results of this kind of behavioral measurement 
provide, first of all, a criterion for performance based upon skills established 
in individual training, as well as a means of testing the efficacy of selection and 
individual training techniques. But performance measurement does not stop 
here. Assessment is also needed of the performance of the teams that operate 
Subsystems, as well as of the performance of the total "team" which operates 
the system as a whole. For this reason, performance measurement takes on an 
additional essential role in the later stages of system development, particularly 
in the system test. . 

Although we have described the events of the design and development 
stages for human components as occurring in order, two other characteristics 
should not be overlooked. The first is the fact that 
s of development for machine and for man occur 
all along the way. As the design for hardware becomes progressively defined 
in detail, changes may be required in the description of tasks which are based 
upon this equipment; these in turn occasion changes or refinements in the task 
analysis, and consequently in the requirements for selection, training, and 
Performance measurement. The interaction can also work the other way. It is 
Not at all uncommon, as the operational uses of the system become more 
firmly described, for some undesirable or infeasible assumption concerning 
job design, selection, or training to be revealed. Such an occurrence may lead 
to alterations in the design of the equipment or to the development of addi- 
tional equipment to perform some previously neglected function. Although 
Figure 0.1 does not show these interactions, they are nevertheless an important 
aspect of the system development process. 3 í 

The second general point to be remarked upon is the occurrence o 
"testing" throughout every stage of development. Many aspects of the design 
Of jobs, for example, depend upon a test to prove their effectiveness in terms 


of the development process 
interactions between the line 
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of human performance; this is often referred to as a "human engineering 
evaluation." Similarly, job aids must often be tried out in a systematic manner 
before their form can be finally decided upon. The testing of selection tech- 
niques is a well-known procedure; the evaluation of training and performance 
measures are also conducted according to well-established methods. lt is 
apparent, therefore, that the process of development is a dynamic one, a 
matter of continuous decision making, testing, and revision, and the simplified 
step-by-step illustration of it contained in Figure 0.1 should not hide this fact. 


System Testing and Operation 


Once the various developmental activities have been carried out to pro- 
vide the men required for the system, it is time to bring together the products 
of this effort and those of the parallel machine-development sequence. The 
System as a whole may now be assembled with completed components of 
hardware and men, and “put into operation." 

The process of putting a system into operation is seldom done all at once. 
Rather, this process itself is composed of several stages which are carried out 
more or less systematically for any given system. These stages are given vari- 
ous names, including "system integration," "shake-down," “system test,” and 
"operational test," among others. Actually, two major kinds of effort can be 
distinguished; they usually proceed simultaneously, and are mutually inter- 
dependent. The first of these is the exercise of the System, to establish and 
refine the interrelationships of its subsystems by means of communication and 
other forms of human interaction; in other words, to accomplish system train- 
ing. The second comprises the set of procedures used to determine what the 
System can do and how well it performs the operations implied by its purposes; 
this is system testing or, more generally, system evaluation. 

It is sometimes Supposed that system testing can be accomplished on the 


equipment alone, but the complexity of modern systems makes it more and 
more obvious that this is a fallaci 


n under such circumstances, there are many 
examples of “test failures" that are directly traceable to human error and 


failing. If the test is to be useful, the occurrence of each failure must be traced 
back to a deficiency which mz 


human beings are supposed 
they are highly skilled. This 


But more than this, the testing of 
leaves a major question yet unanswere. 
operation? Such a question implies the 


a System with highly skilled personnel 
d. How well will the system work in 
need to know the level of the system's 
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performance when in a “normal” environment, including the social environ- 
ment created by the kinds of people who will, over a period of some years, be 
selected and trained to operate the system. System evaluation must therefore 
be conducted under as realistic conditions as possible, as well as under condi- 
tions that permit the taking of analytical measures of its performance. For 
this reason, the final stages of system testing frequently go on for a number 
of years during which the system may actually be said to be “in operational 
use." There is usually no hard and fast line that can be drawn between the 
stage of "system test" and "operational use"; instead, one suffuses imper- 
ceptibly into the other. Accordingly, it is to be expected that system training, 
by means of which increasing efficiency of operation is obtained, and system 
evaluation, which measures the system's performance, will both extend into the 


period of actual usage of the system. 


PLAN OF THE BOOK 


[c] 


The purpose of the book is to provide an account of the various proce- 
dures of psychotechnology used in the development of systems and the 
principles of psychological science on which they are based. In general, each 
chapter provides a description of the technology employed at a particular stage 
of development of the "man" section of the man-machine system, some 
examples of these techniques, and a discussion of their relation to relevant 
aspects of the science of human behavior. The chapters are arranged in the 
order in which problems of development of human components and teams are 
ordinarily encountered during the entire cycle of system development. A brief 
Stage-setting introduction has been provided for each chapter, the purpose of 
which is to relate the information among chapters, as well as to the process of 
system development as a whole. ; R 

In an initial chapter, the nature of psychological effort in the systems 
framework is described, including the ways used to apply psychological knowl- 
edge to the process of system development, and the potentialities of systems as 
vehicles for psychological research. Following this, there are three chapters 
dealing with psychological principles of specific relevance to system design 
and, more basically, to the whole process of development of human com- 
ponents of systems. Chapter 2 describes the nature of human functioning as 
Seen by the psychologist. Chapter 3 relates the capabilities of man to that most 
important modern equipment component, the computer. Chapter 4 discusses 
the capacities and limitations of human functioning. Related to these in its 
relevance to the design stage is Chapter 5, which describes the kinds of 
Psychological knowledge that enter into the design of equipment components 
for a system, and the consequent design requirements for such components. 

The stage of development proper is represented by a set of chapters 
Which follow. These begin with Chapter 6, dealing with the topics of task 


o3 2.5. Ro» b VH iescaren, and to indicate the 
aspects of psychological science which may be expected to yield additional 


facts and principles as the development of the science of behavior proceeds. 
gniz 
it is that we as Psychologists know which can be applied to the design and 


development of Systems, as well as what it is we need to discover in the 
future prosecution of Psychological research. 
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description and task analysis. The logic of procedures used in the selection 
and classification of personnel for Systems is discussed in Chapter 7. Chapter 8 
gives an account of the development of aids to job performance, and the 
rationale on which they are based. Chapter 9 considers the important subject 
of training, and Chapter 10 provides an account of the requirements for train- 
ing devices generated by training programs. Progressing in time to the latter 
end of the phase of system development, we come to a consideration of team 


functions and training in Chapter 11 and of techniques for assessing human 
performance in Chapter 12. 


Consideration is given in two final chapters to the problems encountered 
in using psychological principles and methodology in situations involving the 
testing of total systems. The subject of evaluating system performance is con- 
sidered in Chapter 13. Chapter 14 discusses generally the nature of psycho- 
technology as an approach to System research and development. 

System development is an enterprise of growing importance in our 


society, and of rapidly expanding interest to the psychologist. Psychology as a 
science has a tremendous amount to offer as the basi 
nology of human behavio 


NOW THAT WE HAVE INTRODUCED THE | 
reader to the kinds of activities that take place in 
system development and given some idea of the se- | 
quence of these activities, it is appropriate to set the | 
stage for the unfolding of the details of the whole | 
enterprise in which psychological scientists engage. 
This is the purpose of Chapter 1. 

What is the nature of a system? When one engages in the development 
of technology, the application of psychological principles to developing 
systems, how can these in fact be conceived as “influencing development”? 
Is it possible that this process of development is itself something that can 
become the object of scientific inquiry? The attempt to answer questions such 
as these leads in this chapter to a consideration of some of the. rationale for 
systems and to a view of their essential nature. Systems are described as man- 
made “synthetic” organisms, whose components, subsystems, and interactive 
mechanisms have analogous functions to those of biological organisms, as 
well as a definite life cycle. Men and the organizations they establish as part 
of the total system create the characteristics of stability and continuity which 
are a prominent feature of systems. 

Within this organismic framework, it is possible to conceive of the 
specific activities that form a part of system development—and that are to be 
discussed more fully in subsequent chapters—as interventions which are made 
by system developers initially, and later by system managers, in the on- 
going life processes of the synthetic organism. The effects of such interven- 
tions as selection of human components, their individual and team training, 
and the provision of suitable supports and job environments, can be studied 
and evaluated as a part of the development enterprise. But there is also an 
additional possibility. Because of the nature of a synthetic org 
functioning of its parts and subsystems may be studied directly, 
ing a means of developing systematic accounts of the processes o 
within the system, as well as their effects on tot 
Although the scientific study of systems in this sense has been going on for 
only a relatively few years, it provides at present the lively promise of the 
development of an orderly body of "system theory" which will illuminate the 
operation of a great variety of human organizational enterprises, as well as 
the activities of the individual organism. 

In the sense that it relates the concepts of system and system development 
to the aims of psychology as a science, this chapter is intended to provide a 
setting against which the process of developing the *man" part of man- 
machine systems can be viewed in broad perspective. But the chapter has 
another purpose too: it provides the opportunity for the student of psychology 
to appreciate some of the future possibilities of the system as a point of view 
for the development and use of scientific knowledge about human behavior. 
12 


anism, the 
thus provid- 
f interaction 
al system functioning. 


PSYCHOLOGY AND 
SYSTEM DEVELOPMENT 


John L. Kennedy 


“Man"—what does the word really mean? To me it suggests no mere 
Primeval ape-like creature that walked erect and had hands. To be truly 
human, he must have had the power to reason and the ability to fashion crude 
tools to do his work. 

There is the key, the ability to make tools, as distinct from merely using 
the pointed sticks or sharp stones that lay readily at hand. Such a being, who 
set about shaping the raw materials of nature in a regular pattern to suit his 
needs, was the one worthy to be considered the earliest human. And at last 


we have found him. 
I call him Zinjanthropus, or East Africa Man. He lived more than 


600,000 years ago.’ (LEAKEY, 1960) 


Zinjanthropus and the stones that he fashioned into crude choppers, 
enabling him to add meat to his dict, formed the first simple man-machine 
system. The shaped stones extended his ability to perform the task of skinning 
an animal—a task that he could not accomplish without them. Why system? 
Because the goal is achieved by an interaction between man and device. When 
We add the stones to the man, we design a different organism, man-in-relation- 
to-stones—indeed, a “synthetic” organism. In the ensuing 600,000 years, man 
has created an ever-more complex series of synthetic organisms by the process 
of physical and social invention. He has found ways of extending his individual 
Natural abilities in every direction, so that modern man-machine systems 


1 Quoted by permission of National Geographic Magazine. Copyright 1960, National 


Geographic Magazine. 
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may now consist of hundreds of men and machines covering thousands of 
square miles. 


The Development of Systems 


The synthetic organisms we call man-machine Systems appear to be 
relatively unstable, ad hoc, and contrived when compared to natural organ- 
isms, but they go through processes of development akin to biological evolu- 
tion on the one hand and to the life cycle of a single organism on the other. 
The developmental process similar to biological evolution is that of design 
and redesign of families of man-machine Systems, such as has occurred in the 
"evolution" of the jet aircraft, the atomic submarine, or the high-speed digital 
computer. Possibly the term “design evolution," as opposed to biological 
evolution, might assist in keeping the analogy within bounds. 

When men and machines are assembled into a particular operational 
System, the assembly exhibits development, involving stages similar to birth 
(test in the operational environment), infancy (initial operational capability ). 
maturity (full operation in relation to other systems), and senescence (phasing 
out for obsolescence). Possibly the term “system life cycle” might help in 
distinguishing these developmental Processes in man-machine systems from 
their analogs in the individual biological organism. 

Bonner (1960) points to the trilogy of evolution, genetics, and develop- 
ment in describing biological organisms. It is difficult to conceive of an exact 
analogue for the genetics of synthetic organisms, but an analogous process 
might be found in the sets of plans and decisions resulting in changes in 
system design during the life of a particular design configuration. Thus, in the 
family of piloted aircraft, the progressive changes from B-17 to B-36 could 
be viewed as genetic modifications on t 
while the shift from piloted aircraft to 
evolutionary change. In any case, desi 
now occurring at such a ra 
that they may be indistin 
of system development 
components as given. 


he same general design configuration, 
missiles could be viewed as a major 
gn evolution and system genetics are 
pid rate, as compared to the biological process, 
guishable. This book is primarily devoted to problems 
during the system life Cycle, taking the machine 


and continuity of 
achines creates an 
> individual men. Thus, com- 


What is a System? 


We have discussed complex man-machine systems in 


s lex a gencral way, but 
we have not given either a definition or examples. What ar 


e systems, and how 
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would one recognize such an entity? Synthetic organisms are not as easily 
defined, specified, or recognized as natural organisms because they are not 
bounded by a generally accepted interface, such as the skin. And they have 
the property of being embedded in ever-larger systems. This is not a new 
problem in science, but it warrants some discussion. Astronomers, for example, 
seem to have little conceptual difficulty in defining a solar system, even 
though it is obvious now that a particular solar system is embedded in a 
larger system such as a galaxy, which is embedded in the Milky Way, which 
is embedded in the universe. The definition of system is in a sense arbitrary 
and depends heavily on a priori definition of a task or problem. If the problem 
is to account for the behavior of the planets in the solar system, it is possible, 
at one level of explanation, to consider the interactions between the sun and 
the planets only. It turns out that the concept of gravity will account for many 
of the major phenomena observed. Another problem or task may require the 
definition of a larger system to discover the essential laws to account for the 
observed phenomena. 

If the problem is to descr 
behavior, it may suffice to treat t 
there is much human behavior, such 


ibe, predict, and control individual human 
he individual organism as an entity, But 
as the work humans perform, that cannot 


be described, predicted, or controlled without reference to the machines men 
Use or to the social organization in which individual humans perform these 
tasks. The concept of system, then, implies a goal or purpose, and it implies 
seen components or parts. For the indi- 


interaction and. communication. betw 
vidual human organism, the components are cells and the subsystems are the 
y lex man-machine systems, the com- 


Organs of the body. In the case of comp! 1 1 
Ponents are individual humans and the machines they control and by which 
they are controlled. A : 
Automobile and driver make up a system, a drive-in restaurant is a 
System, a factory is a system, the United States government is a system and 
the United Nations General Assembly is a system, all of them man-made, 
and thus synthetic, organisms. In these examples it should be noted that the 
Progression is from the man-machine system to the organism composed almost 
entirely of men, the human organization. Thus, man-machine systems repre- 
Sent a subclass of an organizations. ] 
a actibus system, the regularities or laws ARA the 
Interactions of the components are sufficiently well known to ies pido 
deal of accurate prediction possible. Many of the mgolntins ps j oe 
9f man-machine systems remain to be discovered; but, "3 . is d dn 
demonstrate, the process of development 1S being studied. cibos ond 
Machine systems are designed and developed to meet the pages o "Po ques 
that technology and organization make possible, on the one han i : a 
aspirations of man to describe, predict, and control nature and other men 


*mand, on the other. System development is characterized by ee 
Compromise between the desired and the possible; each new system dp - 
Physica] science and technology and our capacity to organize to the limit. 
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This discussion of the origins of man-machine systems may be summar- 
ized with a working definition as follows: 4 man-machine system is an 
organization whose components are men and machines, working together to 
achieve a common goal and tied together by a communication network. 

As systems have become larger and more complex, they have created 
serious economic, political, social, psychological, and even moral problems 
in addition to technological problems of engineering design and feasibility. 
But the most important psychological problem relating to systems arises from 
the necessity to organize people, to acquire them as system components, to 
select and classify them, to train them, to keep them working for system goals 
and to bring their performance to a peak to achieve the system goals. 


What Psychologists Do in System Development 


Psychologists carry out research and development on the human com- 
ponents, subsystems, and total system interactions of man-machine systems. 
They give advice, based upon psychological facts and principles, about design 
interventions in systems, about measuring the performance of man-machine 
systems, and about improving the performance of systems. In rare instances, 
they may actually manage the process of development of man-machine 
systems. They are specialists in “human engineering,” personnel selection, 
training, “human factors," and other applied arts connected with the develop- 
ment of systems. The next section of this chapter contains a brief discussion 
of these activities in relation to the concept of systems as developing organisms. 

All these activities lead to the conception of an even more fundamental 
and important enterprise, that of utilizing the concept of man-machine system 
as a framework for the organization and definition of problems of basic 
research on human behavior. The design and conduct of fundamental research 
to extend our understanding of system development is an activity which 
exhibits great promise for the understanding of human behavior as it occurs 
in socially important and productive situations. These possibilities are dis- 
cussed in the latter portion of this chapter, where we consider the man- 
machine system and its development as an arena for psychological research, 
and give an account of the methods employed in such research. Some of the 


major implications of this approach for the definition of psychological prob- 
lems are also considered. 


[gj 


E APPLICATION OF PSYCHOLOGICAL KNOWLEDGE 
TO SYSTEM DEVELOPMENT 


The concept of organismic development has considerable influence on 
the selection and application of interventions in the system life cycle. Inter- 
ventions are the means by which the system psychologist gives effective advice 
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to the system manager. Interventions may range all the way from advice to 
wait for the next developmental stage to the design and application of a 
particular reinforcement procedure at a particular time in the system life cycle. 
The methods developed by psychologists for effective intervention will be 
discussed in a later section. The timing of their application to get maximum 
benefit is still an unsolved problem. In this section we shall relate the kinds 
of intervention used by the psychologist to the stages of system development. 


Designing Equipment 


As machines and devices are more frequently used to extend man's per- 
formance capabilities in systems, the problem of matching men and machines 
arises, particularly in relation to the informational displays and controls of 
machines. Man, as an operator of equipment, is not infinitely adaptable. He 
has inherent limitations as an information processer and as a manipulator of 
equipment (see Chapter 4). These limitations pertain to a wide range of 
human tasks, from simple reaction time to complex decision making. 

The psychological specialty of “human engineering,” or engineering psy- 
chology, emerged during World War II to assist engineering designers in the 
development and choice of displays and controls which would optimize the 
man-machine relationship. Thus, many alternative designs of knobs, hand- 
wheels, joysticks, and the like, for adjusting and manipulating machines have 
been subjected to comparative psychological evaluation by measuring perform- 
ance with them under standardized conditions. Similarily, comparative psycho- 
logical evaluations have been conducted on the design of numerals, letters, 
maps, scales, dials, and so forth, by measuring performance under similar 
and controlled conditions. A Handbook of Human Engineering Data for 
Design Engineers (Kennedy, 1949) is available to assist in the sagen of 
Psychological information and method to the design stage of system develop- 
ment. Excellent reviews of the engineering psychology literature have been 


Provided by Fitts (1958) and by McCormick (1957). 


Division of Functions 


One important consequence of engineering psychology = has been 
a growing concern over the assignment of functions S ss sa he s i 
machines during the design stage. Such a concern pad ae Or Teudire 
limit the performance of the system agris on its 2x n ER ii 
expensive design modifications (retrofit) during the sys a ape à 
Sions about division of functions also interact with the pro eet de de 
of tasks for the human components. The -— ix imn ira dien 
the nonroutine, problem-solving, "troubleshooting, eise jum ori 
Boal seeking, and integrating functions. Machines perfor: 
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upon to extend human capabilities, such as seeing, hearing, moving rapidly, 
lifting, manipulating, memorizing, and the like. Humans in systems have been 
described as “the glue that holds the system together,” but proper allocation 
of functions during the design stage may substantially assist rapid system 
development. The most important point to be made is this: the design problem 
is not how to design a machine—it is how to design an organism. 

Job design for human components is done in several stages. The system 
designer does a rough translation of functions into jobs in estimating numbers 
and categories of people required to man the system. It will be the task of the 
system manager to actually fit the human components provided him by the 
personnel subsystem (see the next section) into operating jobs. Psychological 
research on job design is still in a relatively early stage of development, and 
much of the technology of placing men in jobs is carried out by consultation 


and advice based on experience with other systems or with previous evolu- 
tionary versions of the same system. 


The Personnel Subsystem 


Man-machine systems are developed by three interrelated and interacting 
subsystems of a higher-order system or organization of planners. The first of 
these subsystems is that of machine design, procurement, logistics, and supply: 
the “hardware” subsystem. The second subsystem has to do with the procure- 
ment, selection, classification, training, promotion, and the like, of men: the 
personnel subsystem. The third consists of the operational managers of man- 
machine systems: the management subsystem. 

From the point of view of the higher-order planning system, individual 
men are considered to be rather independent components who might, at one 
time or another, be required to operate in any man-machine system. The 
personnel subsystem concept is based upon certain practical human life-cycle 
considerations. Human life cycles include death, retirement, resignation, 
obsolescence, and forced termination of employment at the exit end of the 
stream. The concept first of all assumes a stream of people flowing into and 
out of the condition of availability to be organized into operating man-machine 
systems. Secondly, it assumes different lengths of time of availability, ranging 
from the “career” to the “short hitch.” Thirdly, it assumes large and important 
individual differences between the human components. Psychologists, for the 
past half-century, have been investigating the range and nature of differences 
between people by devising, administering, and interpreting tests. Chapter 7 
describes the procedures used and the problems encountered in the psycho- 
logical study of individual differences. Many psychologists are engaged in 
these studies. The practical outcome is information of use to managers—that 
is, to the developers of the system’s operations—who are making decisions 
about people in relation to the system’s mission. 

Let us consider the decisions as exemplified in a military personnel 
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subsystem and the nature of the information supplied by psychologists. The 
first decision about a new human component is whether or not he should be 
acquired as a component. The range of general aptitude for military service 
is so wide that some men are eliminated as possible components in any 
military system. The psychological contributions to this decision are the 
general classification test battery, interviews, special aptitude tests, and other 
screening devices. These contributions are validated against some criterion of 
success or failure in performance during military service. 

If the decision is in favor of acquiring the new human component, the 
next decision is whether to put him directly into an operating man-machine 
unit or to send him to a school for specialized training. The psychological 
contributions to this decision again are classification test batteries, interview 
techniques, and special aptitude measures, validated against pass-fail and 
grades in the schools. 

The next decision concerns the kind of training, and again scores on 
Special aptitude tests plus interview information assist the system developer in 
making wise decisions about personnel allocations. An important decision is 
whether or not the component should be trained to become a manager or 
officer, and again psychological tests of general and special aptitude are involved. 

If the individual is to be given specialized training, he is again managed 
by procedures to which psychologists make important contributions (Chap- 
ter 9), The arrangement of experience for most rapid and efficient learning 
(the curriculum) is influenced by psychological research on human learning 
and training. The tests of proficiency which measure success or failure follow- 
ing training are designed by psychological methods (see Chapter 12). 

As the new component gains experience In the military service, many 
cisions are required. Considerations of promotion, 
influenced by scores on qualification tests designed 
ell as by years of experience and kinds of 
isting military managers at all levels of the 
fficient utilization of the human raw material 


additional managerial de 
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at their disposal. 


Training for Operations 
d into jobs, the designer of operations must give 


Consideration to the criteria of system performance. How weil n it m 
Plish the task it was designed for? Will performance be at a satisfactory level? 


If not, what interventions are needed? apes m: gira 
i inni i e evaluatir 

in this decision process, beginning with the probem o l à 
Performance Dus. Chapter 13). If performance is judged ta ws ee 
(and it usually is in the early stages of development), a variety of psy! 


logical remedial techniques are available, depending upon the particular 


'agnosis of this deficiency. 
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If the diagnosis points to an inadequate level of individual knowledge 
and skill (a common diagnosis in the early stages of a system life cycle), 
individual training devices and procedures (Chapter 10) have been designed 
for improving the system-related performance of individual components. 
Psychological research on human learning and training has contributed heavily 
to the design and utilization of such devices and procedures. The Link Trainer 
for developing skill in piloting aircraft is an example of an individual skill- 
training device. A training device for trouble shooting in the maintenance of 
electronic equipment is another example. Training devices at the level of the 


individual may be used to maintain a given level of individual proficiency, 
as well as to achieve it. 


Environmental Supports 


In addition to individual training devices and procedures, individual 
system-related performance may be improved by providing job aids (Chap- 
ter 8), such as manuals, check lists, procedural outlines. Very often, individual 
performance can be aided effectively by the provision of external sources of 
systematized knowledge about the effects of certain decisions or actions on 
other parts of the system. Job aids may also break complex jobs down into 
step-by-step operations, so that the learner does not become completely 
confused in performing the job initially. 


Team Training 


A common diagnosis in the early stages of system development is that 
system performance may be potentially inadequate because of poor teamwork 
in the operating subsystems. Successful performance of the mission of the 
system depends upon the ability of two or more individuals to act in coopera- 
tion as a group. Thus, the psychological techniques arising from research on 
small groups and teams may be applied to system performance by the design 
and use of team training devices and procedures (sec Chapters 10 and 11). 
Examples of such devices are the gunnery team trainers and combat- 
information-center trainers on naval vessels and in shore stations. 


System Training 


Finally, the diagnosis may point to the necessity for coordinated pro- 
cedural training of the whole operating crew, so-called System training (see 
Chapter 11), in which the complete man-machine system is exercised in the 
operating environment or in a simulation of the operating environment. Ex- 
amples of such system training are fleet exercises, air-defense exercises in which 
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friendly aircraft simulate hostile raids, and the "packaged" synthetic system- 
training exercises prepared by simulating operational problems with com- 
puters. Several hundred psychologists are involved in the latter program alone. 

Psychological research on man-machine system training and evaluation 
and on crew development has not kept pace with the applications of these 
techniques, in no small part due to the expense and difficulty of such research. 
Since complete physical simulation is often impractical, a continuing question 
concerns what needs to be simulated and how faithfully the operational 
environment must be matched in order to permit generalization from the 
simulated environment to the operational environment. 


© PSYCHOLOGICAL RESEARCH ON MAN-MACHINE 
SYSTEM DEVELOPMENT 


The modern trend toward the replacement of men and machines for 
many tasks and the consequent requirement for organizing and coordinating 
the work of man has created new demands on psychological knowledge and 
methods. Raben's (1960) recent and excellent survey of operations and 
Systems research literature lists nearly 1000 references to particular research 
Contributions ranging from mathematical analysis to empirical studies of 
Systems. As psychologists continue to make important contributions to the 
man-machine design and development problem, they are learning how to place 
the individual in suitable interaction with other individuals and components of 
the system and how to develop criterion measures arising from the system as a 
whole in addition to the more familiar component criteria of performance. 

Let us take some examples. The invention of radar makes possible a 
tremendous extension of the individual's capability to detect at a distance. 
This increased capability is, however, achieved at a cost. Instead of an 
“object,” with all its information content, the individual man now works with 
a cathode ray tube display in which the object is translated into a dot or 
"blip," moving against a grid of geographical coordinates. The blip may be 
Produced by a variety of circumstances—it is no longer an object but a 
Symbol, and the object-properties (such as size, shape, number, markings, 
identification, and intent) must be added to the symbol by a complicated 
Process of inference, involving communication with other people and organiza- 
tions, Systematic use of redundancy available in the system is required for 
Checking facts and inferences, and a complex chain of decision-making 
Organizes the work of many individuals. Good distance vision on the part of 
the indivi is thus replaced by a comp r 
oreet at id ion ree capability, on which there are severe 
limitations because of the structure and function of the eyes and nervous 
System, with another kind of performance capability that the organization of 
the task now requires. The new capability demands new information to be 
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supplied by other men and it demands the managerial skills of scheduling and 
planning. It may, in the latest systems, require the construction of complex 
computer programs, and it may put excessive demands on the exercise of 
good human judgment in making decisions about variables not included in 
computer programs. But the most important consequence for psychology of 
the man-machine system is that the criteria for evaluating performance now 
go beyond measures of individual performance. A high correlation may be 
found between the visual acuity of the squirrel hunter and his success in 
shooting squirrels, but the possibility of a significant correlation between the 
individual visual acuity of the radar operators with success of the system in 
defense against an attack by intercontinental ballistic missiles seems unlikely. 
Other human functions are substituted for visual acuity as primary deter- 
mining variables, and the performance of the system is now the result of the 
coordinated action of many men and many machines. 

The search for simple, individual variables is further complicated by the 
nature of system tasks. It is possible to maximize the performance of one 
component to the detriment of performance of the whole system. (“The 
operation was successful, but the patient died.") A possible measure of the 
performance of a radar operator might be to report movements on every blip 
on his screen as rapidly as possible. It becomes clear that he contributes most 
effectively to system performance if he can be trained to do "filtering," to 
exercise “judgment” about the requirements of the rest of the system in pacing 
his reporting. In order to do the filtering job effectively, he needs to be kept in 


communication with and reinforced by information about the criteria of 
system performance. 


Criteria of System Performance 


The tasks of systems are often stated in such broad and general terms 
that the psychologist encounters difficulty in task definition and measurement. 
The primary task of the living organism, to stay alive, illustrates the problem. 
Are there measurable degrees of “aliveness”? Is the concept "alive" sufficiently 
operational to provide hardheaded measures of performance? Clinical psy- 
chologists work with the concept of "adjustment," a mysterious balance 
between organism and environment. For many industrial man-machine sys- 
tems, profit-making is often the system criterion, and economists have laid 
down the rules for the computation of the criterion measure as a continuous 
variable using the unit of moncy. For military and government systems, the 
variables and the units in which task performance may be measured have not 
been well defined. “Military worth" and “governmental efficiency” are con- 
cepts that continue to elude the criterion measurer in much the same way that 
"adjustment" eludes the clinician. 

It may be that the ultimate system performance criterion for many large 
systems will remain a matter for the judgment of experts—the interactions 
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may be too complex to measure in any other way than by psychological rating 
methods similar to those used in estimating the popular vote in national elec- 
tions, or by the decisions of the Joint Chiefs of Staff. 

One possible route for the psychologists in search of system criterion 
measures is to define and work with measurable subcriteria. Such a procedure 
requires task analysis and the careful selection of subsystems that will reflect 
the essential interactions of the system. Cost in dollars is such a subsystem 
variable—it cuts across many of the system interactions. The subcriteria of 
System performance are more difficult to specify. New measures that have 
many of the aggregative properties of cost need to be discovered. Aggregated 
measures of output in relation to input, as in the case of the metabolism of 
the living organism, may provide operating criteria for psychological study 
of system performance. Thus Chapman, Kennedy, Newell and Biel (1959) 
were able to measure the "filtering" of information passing through a 30-to-40 
man air defense crew by comparing input information with information for- 
mally acquired by the system; and they were able to measure distribution of 
effort by comparing the number of responses made to important and unimpor- 
tant information by the system as a whole. Such measures are only examples 
of subsystem variables that may be developed as criteria of performance. 


Systems or Components? 


to manageable components is the standard 


The fracturing of systems in 
have seen, a popular method has been 


Way of dealing with the entity. As we ] 
to separate men and machine components and to define a personnel subsystem 


ànd a machine or hardware subsystem. Such a procedure begs the question of 
total system performance, permitting the psychologist to concentrate on people 
and their properties and the engineer to concentrate on machines and their 
Properties. The problem of integration of men and machines is left to the skill 
and ingenuity of the system manager. Criterion measures for the personnel 
Subsystem tend to be grades in special schools or training programs, while 
criterion measures for machine subsystems tend to be the subgoals of “flying 


higher and faster.” 
By defining a personnel 
Pursue such problems as the measurem 


and skills, motivation, and personality 
Question of operational system performance. There are many advantages to 


Such a procedure. The primary advantage is the casy accessibility of a inion 
Measure such as the performance of people judged in training, against wider 
to validate individual test scores. A tremendous amount of research has been 
done by this means (see Chapter 7) A second advantage is that psycholo- 
gists are not required to understand the equipment environment provided by 
the Operational system. The principal method for improving system perform- 
ance is selection and classification of the human components into aptitude 


subsystem, psychologists are often able to 
ent of individual aptitudes, capacities 
variables quite independently of the 


23. = Psychology and System Development 


categories on the one hand and the development of skill and experience 
categories on the other. Thus the new human system component is placed ina 
general aptitude category by psychological tests and then given specialized 
training (if his general aptitude is sufficiently high) to prepare him for a skill 
category. Operational systems are manned by a “table of organization" which 
attempts to specify the numbers and skills of people required to achieve the 
system goal. The system managers are left with the task of fitting these com- 
ponents into "jobs," organizing their efforts, defining and reinforcing thc 
system goal, evaluating performance; in short, the integrating task. The science 
of psychology can better assist the manager with the integrating task when 
psychologists begin to study man-machine systems as developing, interacting 
organisms. 

Why should psychologists study man-machine system development? 
Development or directed change of behavior in time is the most characteristic 
property of systems, as it is of living organisms. The study of development in 
living organisms has shown that it follows an orderly sequence or pattern, the 
developmental schedule (Carmichael, 1951). In the prenatal period, structure 
and function unfold in a predictable sequence under the control of growth 
processes, hormones, and environmental influences. The variables influencing 
the rate of growth are becoming known and the growth rate may be changed 
by manipulating these variables. After birth, the variables of the environment 
become more active determiners of behavior, but gene-determined growth 
processes continue to create the inevitable stages of infancy, childhood, ado- 
lescence, maturity, and senescence. The managers of man-machine systems 
deal with a growth process analogous to that of the living organism. Systems 
go through a design and assembly stage resembling the period of gestation. 
Systems are "born" when they become operational in the environment for 
which they were designed. They go through stages like infancy, childhood, 
adolescence, maturity, and old age. Weiner (1960) has described rough 
developmental stages for a particular man-machine system. 

An answer to the question of why system development should be studied 
is that similar principles, laws, and rules may be discovered for the design 
and management of man-machine systems to those which have appeared from 
the study of biological development. Characteristic managerial problems 
appear to have their biological analogues. For example, the manager may be 
concerned with a high rate of personnel turnover and its effect on the develop- 
ment and maintenance of operational capability of the man-machine system. 
This is a problem like that posed by the biological grafting procedure, in 
which it has been demonstrated that the success of the graft of new tissuc 
depends, among many other variables, on the stage of development of thc 
organism. Or the manager may be concerned with the problem of a slow rate 
of growth in performance capability and may wish for some technique for 
speeding it up. The biological counterpart, of course, is the technique of 
injection of hormones to speed up biological growth. The psychological 
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methods for influencing rate of psychological growth consist of the design and 
application of interventions, such as special training techniques, reinforcement 
procedures, and motivational procedures. The manager may be forced to 
recognize that techniques appropriate to an early developmental stage of the 
System are no longer desirable or effective at a later stage. A managerial art, 
related to the science of development and growth, may very well replace 
"scientific management," which tends to be based upon simple component 
measures of "efficiency." 

The research psychologist who is involved with systems must also recog- 
nize the overwhelming importance of development in choosing performance 
Criteria and in measuring performance. Performance measures taken at a 
given stage of development should not be viewed as necessarily representative 


of the "potential" of the system. Without some indication of the developmental 
t difficult to interpret. Possibly a 


problem! The IQ does have 
a developmental measure, 


Stage, such performance measures are mos 
measure of “system IQ" would help to solve this 
the property of relating present performance to 
chronological age. 


System Theory 


Although much theoretical work has been done on biological systems, no 
really comprehensive “map” for describing the growth of synthetic organisms, 
for determining how to combine human capabilities with machine capabilities 
to achieve system goals, is now available. 

Of particular difficulty to system theory are the many criteria which are 
necessarily used in evaluating system effectiveness. Typical criteria in cur- 
rent use are system accuracy and reliability, flexibility, autonomy, reaction 
time, and rate of response, resources utilized, enemy resources neutralized, 
resistance to obsolescence, safety, ability to recover from catastrophe, and 
so forth, Even ethical and political criteria must somehow be taken into 
account, Although it may be difficult to find common denominators for 
these multiple criteria, some method is needed to deal conceptually with 
their interrelationships. And research on human performance must be at 
least guided, perhaps controlled, by. known or expected relationships 
between system criteria and the sub-criteria of human performance.” (Re- 

search Group in Psychology and the Social Sciences, 1960.) 
ect and immediate benefit to the man-machine 
he conditions under which the human com- 
ving system performance, on how to 


Of possibly more dir 
System manager is research on t 


Ponents of systems invent Ways of impro à 
make the system as à whole learn to use its resources, both human and 


machine, more effectively. Much more rescarch needs to be done on a general 
hypothesis about speeding up system development in its later stages toward 
Maturity. The hypothesis or principle is: Train the team as a whole in an 
adequately simulated environment and give it immediate knowledge of results. 


? Quoted by permission of the Research Group in Psychology and the Social Sciences. 
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E METHODS USED IN RESEARCH ON SYSTEM DEVELOPMENT 


The methods used by psychologists in studying system development do 
not differ in any fundamental way, except possibly in scope, from those used 
in the study of other psychological problems involving human behavior. Let us 
consider how these methods are applied at the several stages of system 
development, beginning with the "prenatal" stage. At this stage, a man- 
machine system consists of all the individuals who might become components 
of an operating system. 

As we have seen, the psychometric procedures of selection and classifica- 
tion on the basis of aptitude are applied to sort individuals into aptitude, skill, 
and knowledge categories. A standard methodology for selection of test items, 
test administration, recording of behavior, and analysis of data is available 
and generally agreed upon. The methods of selecting and classifying on the 
basis of skill often involve the development and use of special apparatus. 
Knowledge classification again returns to the psychometric procedures of 
design, administration, and interpretation of achievement tests of performance. 

Psychological methods for providing individual knowledge instruction and 
training at the general level are also well established, if not well understood. 
Classroom instruction tends to be evaluated by the verbal or written perform- 
ance test of “memory” for the material studied. These procedures are thought 
to “condition” or prepare the individual for his eventual role in system 
operations but the criteria of performance appear to be cither excessively 
Subjective, based upon ratings of instructors, or excessively objective, like 
scores on multiple-choice tests, the connection of which to system performance 
is often remote. 

When the man-machine system is born, it encounters the operational 
environment with a relatively fixed set of human components and a relatively 
fixed set of machine components. The problem now becomes one of develop- 
ment of the operational crew against measures of system performance. What 
psychological methods should now be used? 

It seems clear that crews (see Chapters 10 and 11) need to be studied 
over long periods of time under a variety of environmental conditions, ranging 
from routine operations to times of maximum stress, The appropriate methods 
shift from those characteristic of the field of psychometrics and traditional 
experimental psychology of the individual (aptitude, skill, and knowledge test- 
ing) to the methods developed by social psychologists for the study of groups 
and organizations. The organism under investigation changes and the tradi- 
tional methodology of individual psychology must accommodate to the change. 

We may observe, however, that psychological methods of experimentation 
are applicable to the synthetic organism by reference to a particular set of 
experiments on system development (Chapman and Kennedy, 1956) per- 


formed in a laboratory which was specially designed to permit the application 
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of these methods. The general hypothesis tested by these experiments was that 
the psychological techniques used so successfull in inducing rapid and 
efficient learning and adaptation in individual living organisms would transfer 
successfuly to guide the rapid development of synthetic organisms. This 
hypothesis was essentially confirmed, although adaptations of standard psycho- 
logical methods were required. Let us use the categories of methods in psy- 
chology devised by Andrews (1948) in describing them. 

1. Methods of selecting the individuals to be studied. Four crews were 
selected to operate a simulated man-machine system requiring 30 to 40 men. 
A first “shakedown” run was conducted with a paid crew of college students, 
selected on the basis of availability, some aptitude test scores, and willingness 
to work part time in the laboratory for a school term. The remaining crews 
were obtained through the Air Force personnel subsystem from operating Air 
Defense sites and were released for six-week periods of temporary duty at the 
laboratory. Air Force rank, skill-category, and aptitude level information were 
available, but the problem of job design and assignment was turned over to 
the commanding officer or manager of the crew. The experimenters wished to 
Observe the changes in job assignment as a function of crew development and 
environmental “stress” applied in the experiments. Such crew selection proce- 
dures ensured that new crews would be under developmental study with a 
minimum of interference from the experimenters. . 

2. Methods of controlling extraneous stimulation during the investiga- 
tion. These experiments were performed in a special laboratory where a good 
deal of the working environment was under the control of the experimenters. 
For example, telephone calls to subjects, except those of emergency nature, 
were excluded. Interaction between the crew and experimenters was kept to a 
bare minimum and was conducted through formal channels. no 

3. Methods of instructing or directing the attention of individuals under 
investigation. Since "set" is such an important determiner of human behavior, 
Many special techniques were devised to establish the appropriate level of 
information about the purpose of the experiments, to explain the nature of 
the system goal, and to foster a developmental point of view. Special lectures, 
instruction sheets, and memoranda were devised to create the desired conditions. 

4. Methods of presenting the stimuli. In these experiments, the stimuli 
Were presented as a total air picture to a variety of simulated radar scopes. 
In addition, written and verbal channels were used to convey related infor- 
Mation to complete the total operational environment of the crew. Stimulus 
Presentation was controlled by automatic timing devices and by a written 


“script” for the verbal inputs. 


istering intermedi ily change. A major reason 
5. Methods of registering interme diate bodily change aj 


for choosing the air-defense direction center as an example of — 
System is that its internal workings are open to the “measurement snack 
Mediate bodily changes"—that is, the “intervening variables n 2 ae 
are accessible. By strategic placement of microphones and recorders and by 
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requiring the synthetic organism to keep records of its internal actions, it is 
theoretically possible to follow each item of input information through the 
system to output without serious interaction between recording method and 
data. Observers may also record crucial data about internal variables because 
the organism is spread out and accessible. 

6. Methods of observing and recording overt responses to stimulation. 
The flow of verbal interaction at many points was recorded on voice recorders 
for later analysis. Observers with headphones could also listen to verbal 
behavior at almost any place in the system. A method of coding verbal data 
into punched cards was worked out, so that specially trained observers could 
code input, internal actions, and output data during operations. 

7. Methods of analyzing and synthesizing the data from an investigation. 
Two phases of data analysis and synthesis were utilized. The first concerned 
the information about performance which was routinely furnished to the 
commanding officer for discussion with the crew immediately after a period of 
operations. Rapid data analysis methods, involving the punched cards 
described above and a computer, were designed to give immediate knowledge 
of results to the crew. The second phase of data analysis involved listening to 
and taking information from recordings and relating this information to input 
and output data. 

The major problem encountered in applying these methods was the sheer 
volume of information—some 12,000 hours of recordings of verbal data and 
some 60 file drawers full of supporting information. With the increasing 
availability and lower cost of high-speed computers for psychological research 
(compare Chapter 3), however, such mountains of data will not in the future 
Tepresent as serious a problem as is currently the case. 

This rather lengthy example illustrates the feasibility of applying the 
methods of psychology to the problem of system development at all stages. 
What kinds of outcomes and implications can be perceived for conducting 
psychological research on the development of man-machine systems? What 
psychological variables are given importance from the system development 
point of view? What new insights for psychological theory may emerge from 
such studies? Let us turn to these questions in the following section. 


[5] 


AN EVALUATION OF SYSTEM DEVELOPMENT 
AS A PSYCHOLOGICAL PROBLEM 


A proper evaluation of system development cannot be made without 
reference to its relatively short and stormy history. World War I started 
psychologists on a long and profitable development of psychological testing as 
a tool for assisting personnel managers in making decisions about the apti- 
tudes, capabilities, and limitations of people to perform in any organization 


or system. World War II created the psychological problem of the 


design of 
equipment for maximum efficiency 


of human use. Some of the equipment 
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invented and utilized during World War II began to require complex and 
coordinated behavior on the part of large groups of men. With the invention 
of radar, for example, new levels of intelligence could be achieved concerning 
enemy plans and actions if only the information could be collected and inter- 
preted rapidly enough. This requirement resulted in the invention of special 
information-collecting and -processing centers, variously known as combat 
information centers, air defense control centers, and the like. Psychological 
study of large interacting man-machine systems really began with problems 
of equipment design for human use and the layout of these information- 
processing centers during and after World War II. The managerial problem 
of organizing the human components and developing their performance to 
satisfy the many criteria described above has only gradually emerged as a 
Problem for psychological research. 

The initial reaction to the problem on the part of both psychologists and 
managers was hostility. The managers of these new organisms were hostile to 
the notion of research on system development, because it seemed to call into 
question traditional management methods. The psychologists were hostile 
because such research seemed to go beyond the boundaries of "accepted" 
Psychological investigation and methodology. Many false starts were made, 
laboratories and research teams were established, only to discover that the 
Problems of system development were difficult, the cost in relation to tradi- 
tional psychological research was astronomic, and gains in understanding 
System development were slow in coming. 

But the rapid development of machines demanding more and more 
Organization of the human components of systems continues to pose the 
System development problem—new systems demand new managerial insights 
and methods, and psychology is the source of such methods. The practical 
Outcomes of better system performance sooner and at lower cost, both in 
People and machines, are too important to ignore. . . 

The major implication of the system. development point of view for 
PSychology seems to be that psychological methods and techniques must be 
expanded to encompass the managerial problems encountered when men and 
Machines are organized into complex systems. The traditional way of deal- 
ing with people in systems by means of an externally organized "personnel 
Subsystem" does not force the psychologist to confront the criteria of total 
System performance and the developmental processes involved in achieving 


System goals. 
The kinds of human beh 


Ment problem may be listed as follows: m Mu 
l. Learning, and the transfer of skills acquired in one situation to other 


Situations. Much rescarch on learning has been done on the individual human 
learner, This research needs to be expanded to include the learning and 
Adaptation of human groups and organizations, using system measures of 
Performance, 

2. Planning and programming. 


avior given importance by the system develop- 


These managerial skills involve the 
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ability to anticipate, forecast and take corrective action on situations before 
they arise. Planning and programming represent one of the highest levels of 
human functioning. 

3. Inventing and problem solving. The technological revolution has put 
a premium on invention of machines and machine processes. Invention in the 
area of management techniques and methods has lagged far behind. The 
hypothesis may be entertained that every human individual possesses the 
capacity to invent at some useful level. Man-machine systems, by managerial 
techniques, might be able to "improve themselves" if this inventive ability 
could be brought to bear on system problems of coordination and development. 

4. Communicating. At the heart of the problem of coordination and 
successful group effort lies the process of communication, the skill of when to 
say what to whom. Complex man-machine systems live on a stream of 
communication, both formal and informal. Studies of rates, timing, and the 
content of communication that characterize "successful," rapidly developing 
systems as opposed to unsuccessful systems would represent a first attack on 
the function of communicating in man-machine Systems. 

The question was raised carlier about possible implications for psycho- 
logical theory arising from the study of system development. In many theoret- 
ical formulations of behavior, assumptions about "intervening variables" or 
hypothetical constructs of one kind or another seem to be necessary. These 
mysterious entities arise because the individual living organism presents prob- 
lems of access to its internal workings. By defining and working with synthetic 
organisms, the problem of access is ameliorated, if not solved, since the 
synthetic organism is "spread out" and its boundaries are not as definite as the 
skin. The possibility exists that the study of the internal workings of synthetic 


organisms may give us hints about the mechanisms underlying the behavior 
of individual living organisms. 


[2] 


SYSTEM DEVELOPMENT AS AN ARENA 
FOR PSYCHOLOGICAL RESEARCH 


As we have seen, man's behavior plays a part in all phases of system 
development, both in design evolution and in the system life cycle, where 
man functions as planner, designer, manager, and operator. System develop- 
ment provides a “rich” environment in which to study human behavior, from 
basic sensory processes to the most complex skills (see Chapter 2). Because 
system development is an important social and economic problem, the 
PSychologist may find resources available to support kinds and levels of 
investigation extremely difficult to perform in other settings. 

It is often possible, with some ingenuity, 
laboratories, already rather fully equipped, 
ting man-machine Systems. Thus, 


to establish experimental 
in the field installations of opera- 
an air traffic control center, an air defense 
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center, a submarine, a destroyer. a computing center, and a factory, are all 
laboratories in which interesting human behavior is taking place. Simulation 
methods, such as those described in Chapter 11, now enable the psychologist 
to gain enough control of many operational environments so that the field 
installation does in fact become a laboratory. 

These “field laboratories" differ from the standard academic psycho- 
logical laboratories in a number of important ways. In the first place, the field 
installation has a mission of its own, to which psychological experimentation 
must be accommodated. Quite often, however, mission and experiment are not 
in conflict, particularly when the experiment is designed to study the perform- 
ance of the mission. The psychologist working in such field laboratories must 
expect frustrations if he aspires to elegance of experimental designs and must 
exercise considerable ingenuity in "working around" the mission. The main 
Offsetting advantage is, of course, the relative ease in obtaining subjects or 
Observers, particularly when the research is related to the particular mission. 
Such field-laboratory work makes possible the extension of academic research, 
using the college undergraduate as subject, to greater ranges of aptitude, skill, 
age and motivation. 

The psychologist working in the field laboratory is often required to 
“stretch” standard scientific methodology to encompass the problems he finds. 
Very often he is unable to define an independent variable and a dependent 
variable and hold all other variables constant, as he may have been taught to 
do, and he may therefore be driven to the growing field of multivariate analysis 
to find methodology suited to the problem. 

In addition to the field laboratory, several more traditional psychological 
laboratories for the study of system development have been in operation for 
Some years. The former Systems Research Laboratory at the RAND Cor- 
Poration, the Logistics Simulation Laboratory of the RAND Corporation, the 
Human Factors Laboratory of the System Development Corporation, and the 
System Simulation Research Laboratory of the System Development Corpora- 
tion are examples of such special laboratories. — ] 

In these installations the machine aspects of man-machine systems or 
Components of systems are often represented by programming a computer to 
Present operational problems to individuals or groups of men. These special 
laboratories generally concentrate on the performance of people and the ways 
in which their performance in achieving system goals with machines may 


be improved. 
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PLANNING FOR THE DEVELOPMENT OF A 
system begins with a series of decisions regarding the 
functions to be performed by various parts of the system 
in their subordinate contributions to a total complex 
which will accomplish system goals. On the side of 
equipment, these decisions are based upon current hard- 
ware technology and upon estimates of the feasibility 
of new combinations of machine functions. To a considerable degree, also, 
they are founded upon estimates of what man can do, since the subsystems 
which result from these plans almost invariably include human operators. 
Assumptions are made and conclusions are drawn about human functions in 
systems, which ideally involve thorough and orderly considerations of the 
comparative capabilities of man and machine. . 

The chapter which follows attempts to describe and categorize the basic 
types of human functions which are known and differentiated by psychologists. 
These are the "information-processing" functions of man, which are generally 
agreed to divide themselves into the three major categories of sensing, per- 
ceiving, and thinking. These are here given the names of sensing, identifying, 
and interpreting, in order to more certainly imply their operational definitions. 
The attempt is made to represent these human functions and their interrela- 
tionships by means of flow diagrams, and to provide a rationale for doing so. 
Using such a representation, it becomes possible to consider the bases for 
limitations in human functioning and to draw some instructive comparisons 
and contrasts between the functions of man and machine. 

The early decisions Which assign functions to man and to machine can 
be seen to affect the entire course of development of the “man” portion of a 
man-machine system. The functions required of man are reflected immediately 
in the description of the Specific tasks he must perform, and in the design of 


jobs he must fill. These in turn determine what criteria will be used to select 
suitable human beings fi 


training, and to furnish t 
Later still, the choice of 


of the total performance of the System. 

This view of man's functions in relation to those of machines poses a 
number of interesting questions to be answered by scientific research on 
human behavior, some of which are described here. In a more general sense, 
sting the possibility that the classifica- 
cture within which research questions 
th considerable usefulness. 
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HUMAN FUNCTIONS IN SYSTEMS 


Robert M. Gagné 


A basic assumption of the system development point of view is that man 
can be considered as one of the major components of a total system. This view 
is opposed to the notion that man simply plans, buys, develops, and uses a 
System once it is built. Any reasonably complex system requires a true inter- 
action between man and the other parts of the system, which may be machines, 
Other men, or combinations of these. Some way must therefore be found for 
thinking about the functions of machines and the functions of men within a 
framework which makes possible the relation of these two kinds of functions 
to common goals—that is, to system goals. Even in a system as familiar and as 
relatively simple as the automobile, it is easy to see that the goal of transport- 
ing passengers over roads requires not only the functions of the machine itself 
but also a considerable variety of human functions performed by the operator, 


as well as auxiliary functions performed by such people as traffic policemen 


and filling-station attendants. The design of a system which is to be successful 


in achieving some socially defined purpose requires thorough and continued 
Consideration of the interacting functions of both men and machines. 

_ What we propose to do in this chapter is to describe some of the func- 
tions of man and to show how these can be related to the functions of the 
Machine environment in which man is placed as part of a system. We shall 
attempt to do this by developing and using a language that relates input for 
the human being to his output, which in turn becomes an input to some other 


Portion of the system. In other words, in dealing with man’s functions, we 


Shall be identifying the kinds of transformation which an input undergoes in 
g 


Order to be reflected as a human output. The nature of such a language is 
fundamentally not very different from the kind used by equipment designers 
Who speak of machine functions as "coding," "searching, and "storing. 
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However, we shall aim to be as precise as possible about definitions, using 
"e i ” language. 

ee be concerned with the description of human functions 
in systems? The reason is not far to seek and will be abundantly illustrated in 
this and later chapters. Knowing what human beings must accomplish in 
systems, to express these accomplishments as functions makes possible the 
accurate description of man’s required capabilities. These accounts in turn 
permit inferences to be made concerning the various techniques needed to 
develop these human capabilities, whether they be matters of selecting men, 
training them, or providing them with adequate environments for their work. 
These techniques constitute the subjects of later chapters. In a fundamental 
sense, they all rest upon decisions that concern the nature of human function- 
ing and its relation to that of machines. 

We shall attempt to develop this description of the varieties of human 
functioning in the context of a fictitious system of increasing complexity, whose 
goals can be related to those of other existing systems. We shall also find it 
informative to relate the human functions we are able to identify to those 
with which the science of psychology has traditionally dealt. As will be 
seen, placing man in a system context makes possible a novel and instruc- 


tive view of some of the categories of human functioning with which psy- 
chology is concerned. 


Man in the Man-Machine System 


In order to develop definitions of man’s functioning in systems, we need 
first to set the stage. In a very general way, we need to describe what a system 
is and how man fits into it. 

A system is developed to fulfi 
Its purpose may be to protect agai 
destroy an enemy in wartime. Bu 
military enterprises, They may hav 
those of an airport-to-cit 
check-cashing system. 

In order to fulfill 


Il some human purpose or intended use. 
nst enemy military attack or to harass Or 
t systems, of course, are not confined to 


s, breakdowns, 
and even to virtual system abandonment, No 


—Robert M. Gagné = 


system is complete until it can be shown to operate within a total setting that 
includes human beings; no system can truly be said to be successful until its 
operational effectiveness is demonstrated. The best system development is that 
which includes consideration of system operation (rather than merely hard- 
ware operability) from the very beginning of system design. 

There have been several attempts to describe the place of the man- 
machine subsystem in a total system, as well as the place of the human 
component within a subsystem. One of these is by Taylor (1957), whose con- 
ception of man’s role in a system is shown in Figure 2.1. Here, the human 
Operator is pictured as a data transmission and processing link inserted 
between the displays and controls of a machine. An input is transformed by 
certain mechanisms into a signal, which is displayed as a pointer reading, a 
pattern of lights, an oscilloscope wave form, or the like. This information is 
read by the human operator and transformed into responses—the pushing of 
Switches, the moving of control handles, and so on. These in turn generate 
control signals which are transformed by mechanisms into system outputs. 


Man 


Controls 


Displays 


Mechanisms 


Fd EY Output 
Input 


Figure 2.1. The place of man in the man-machine system (after Taylor, 


1957). 


In some systems (the closed-loop type) this output works back upon the 
displays, so that they reflect the human operator s response; in others ere 
loop type) it does not. As Taylor points out, if the designing of the mechanica 


Components of a system is to be done successfully, it must proceed in pa 
Way as to take into full account the characteristics of man and his role in 


the system, 


ioni many points and in 
Man's functionin y P 


g enters into complex systems at 
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many particular ways. Furthermore, the display of information, the controls 
to which the individual responds, and the mechanisms which provide the 
transformations for these components of the system are of considerable 
variety. Accordingly, we need to recognize at once that Taylor's generalized 
picture, while it indicates man's position as a system component, does not 
provide the means for a detailed analysis of the varicty of human functions. 
It would be a mistake to think that because man typically “occupies a space 
between machine displays and controls, his functioning can be related in a 
constant set of ways to such inputs and outputs. The fact is, neither the input 
nor the output by themselves will tell us the nature of man’s functioning. For 
there are different kinds of transformations which may be performed (by the 
human nervous system) in turning inputs into outputs. 

To understand the variety of these transformations, we must first recog- 
nize that there are internal functional units in man, some of which may be 
active or inactive (that is, shunted out or shunted in) for any particular kind 
of functioning which the system may require. The science of psychology may 
be considered as having the purpose of discovering and defining the functions 
of these internal units, as well as their interrelationships. The extent to which 
this process has been successful to date will determine the adequacy with 
which the functions of these internal units can currently be described. 

Although we shall be concerned with describing the functioning of these 
units throughout the chapter, perhaps it will be desirable at the outset to 
illustrate the point that human functioning can be understood in terms of what 


kinds of internal mechanisms are activated by an input and what kinds are 
shunted out in any particular instance. 


Suppose that a system has availa 
display. On the face of this sco 
adjusted in amplitude to a pa 
two fine horizontal lines on a t 


ble an oscilloscope as a basic unit for 
pe appears a 60-cycle wave form which can be 
rticular size, the required size being given by 


ransparent overlay placed against the tube face. 
Let us assume that the system requires the amplitude of the wave to be 


determined within very close tolerances before further operation of the 
system can take place. Following this, what happens is that an external signal 
distorts the form of the wave, and it is the human operator’s task to “report” 
the nature of these distortions by pushing one of five buttons. If there is an 
amplitude distortion (vertical displacement from the overlay markers), he 
pushes button 1; if there is a frequency distortion (horizontal displacement 
from other markers), he pushes button 2; if there are additional frequencies 
present (irregular wave pattern), he pushes button 3; and so on. 

Now, saving certain details for later consideration, let us consider the 
difference between what the operator does in “getting the equipment ready to 
operate” and what he does in “operating.” Actually, he is utilizing two different 
functions in the two cases, even though the display may be the same in both. 

In placing the equipment into proper operating condition, the human 
operator is making use of the function of sensing. That is to say, he is using 
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his visual receptors, nervous system, and effectors simply to "report" (to a 
machine, typically) the presence or absence of a difference in physical energy. 
In this case, the physical difference being reported is the coincidence of two 
pons (or small areas), each of which lies along a narrow band of light (a 

line") which makes an abrupt gradient of intensity with its surround. The 
Operator is exercising a function called visual acuity, a particular name for one 
of his sensing functions. 

. It may be noted, however, that the operator is able to do much more than 
this, even within the equipment-readying stage of operation we are consider- 
ing. He is perfectly capable, for example, of making an output which reports 
the amplitude of the wave along some scale such as millimeters or volts. He 
is able to tell us the color of the wave form, to estimate whether it is bright 
enough, or whether it has a regular appearance, and many other things. Why 
does he not do all these things in this situation? There is no mystery to this 
question at all: he does not simply because we have not told him to (or 
perhaps told him not to). But this means we must recognize that there is more 
to the matter of input than simply the presence of a display. In order to get 
the output required by the system, the operator must be provided with a set 
of instructions. 

One basic purpose of these instr 
determine which functions "higher up 
The combination of the oscilloscope display and the instructions is what 
determines the output that will be made. The instructions say to the operator, 
in effect: “Report coincidence between a set of lines. Do not report their 
Shape, or size, or brightness, or regularity, or meaning, or anything else." 
Thus it is apparent that the effect of presentation of the oscilloscope display 
Plus a particular set of instructions is to put into operation a particular kind 
of human function, sensing, and to shunt out other kinds of functions of which 
the human operator is capable. The situa 

Now let us contrast this elementary kind of behavior with what occurs 
When the equipment is being operated rather than merely turned on. In this 


uctions, we are now able to see, is to 
" than sensing are to be shunted out. 


— > Output 


is — sese | 


Sensing 
instructions (Shunt) 
Figure 2.2. The combination of a display and “sensing instructions” 
hich reports the presence 


as input to a human operator produces an output w à - 
or absence of a difference in physical energy, the function of sensing, and 


shunts out other functions of which the operator is capable. Compare with 


Figure 2.3. 
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case the human operator must function in quite a different way. He must 
provide five different output responses (press one of five buttons) whenever a 
particular kind of deformation of the sine wave appears, whether it is a change 
in the horizontal dimension produced by variation in frequency, in the vertical 
dimension (amplitude), or in one of several other types. In other words he 
must identify five different classes of patterns appearing on the Scope. 

When the operator is engaged in this function of identifying, has the 
internal mechanism for sensing been shunted out? Of course it has not, because 
the physical differences which determine the existence of classes of stimuli to 
which the operator responds must be sensed in order for identification to take 
place. Thus we see that, on the input end, human functions have a hierarchical 
arrangement. The use of a function like identifying requires that a function 
lower in the hierarchy, sensing, be put in operation as well, 

Again, however, we can see that certain even higher functions have 
indeed been shunted out when we ask for identification. For example, the 
operator may be capable of telling us that a deformation of the sine wave in 
the vertical dimension “means” a change in amplitude. But we have not asked 
him that; we merely asked him to press a button indicating the presence of a 
particular class of change in wave shape. Or again, he may be capable of 
interpreting this kind of change as indicating the presence of a type of remote 
signal received at the other end of the system. This too would be a function 
at a higher level than identification, and we shall consider this later on. At the 
moment, our point is simply to contrast the situation of Figure 2.2, depicting 
the sensing function, with Figure 2.3, showing identification. Again in this 
figure we see that one of the primary effects of instructions is to keep the 
human being functioning at the proper level and to shunt out other, higher- 
level functions. 

Another internal mechanism must be added at this level, too, and that is 
memory. For we know that without some kind of long- 
representations of the five different changes in wave shapes, 


e Memory 


Display ———1 Sensing [>] eniti, 


Identifying 


term storage of 
the achievement 
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instructions (Shunt) 
Figure 2.3. The function of 
play as shown in Figure 2.2. This 
tions” 


identifying in response to the same dis- 
function requires “identification instruc- 
and a memory which provides an internal set of standards for the 
wave shapes to be identified by different outputs. Again, the instructions 


operate to shunt out higher-level functions of which the human Operator is 
capable. 
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of five different outputs would be impossible. Instructions alone will not do 
the job. To be sure, we can describe to the individual what he is expected to 
do, by means of instructions, but he will nevertheless not be able to do it by 
this means alone. He must have an internal means of matching the external 
display to one of five classes in order that he can make five different responses, 
as required in our example. This means he must have previously acquired the 
“representative shapes” in his memory, by means of learning preceding the 
occasion when he tackles the job of operating his equipment. And this provides 
the basic reason for training, as well as for the crucial part it plays in the 
system development process. 

Now, this description of two examples of human functioning has not 
involved very high-powered psychology; we are well aware of that. We have, 
in fact, been describing the functions of sensing and perceiving, which have 
been studied by psychologists for many years. But the purpose of our account 
has not been to review basic principles. Rather, it has been to show that the 
fundamental operations of psychologists in describing the functions of man’s 
behavior are, in fact, very much like the operations of a design engineer in 
describing a machine. Psychologists make the same kinds of inferences about 
human behavior as designers do of machines, and they are based upon the 
same kinds of objectively defined operations. It should therefore be quite easy 
for the psychologist to communicate to others the nature of human functions, 
provided he remembers to state clearly what input conditions must be met 
and what the output achieves (as an input to the next unit of the system). In 
Our further delineation of human functions, we shall find it useful to refer back 
to the conceptions developed in these relatively simple examples. 


HUMAN FUNCTIONS 


Now let us undertake to view the nature and variety of human functions 
Which are employed in systems. We shall do this by using illustrations of 
typical human tasks that occur in actual systems, such as telephone systems, 
aircraft, and missile systems, traffic-control systems, and air-defense systems. 

Ut the major technique we shall use will be one of describing à variety of 
tasks which could be performed by a single subsystem of a a system. 
i i ing any of a number of purposes, 
© can imagine the system as a whole as having i Tpos 
"t we shall concentrate on the basic unit ofa ponies Ripe s origin 
imi in Fi i we sha 
Similar to the one depicted in Figure 2.1). In this way we t 
Several different kinds of tasks for the subsystem, progressing aes oe snes 
to the complex, with the expectation that these will permit He » ra eds 
© varieties of human functions and the conditions under which they operate. 
i uc - 
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Connected by either human or machine “communication links. - d we 
se a particular system, we might miss some of the kinds of human function- 
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ing we want to describe. Accordingly, the subsystem becomes the initial focus 
of interest for our discussion. 

Imagine, then, that we have a subsystem which presents to the human 
operator certain information in the form of a display panel containing one or 
more "pictures" of various sorts. Such pictures may include a dial with a 
graduated scale and a pointer, an oscilloscope with various forms of visual 
information, and an aerial photograph of selected portions of the earth's 
surface. The operator, seated at a console containing these displays, has at his 
fingertips several switches, a small joystick mounted in a universal joint, and a 
means of verbal communication (telephone) with another human being who 
constitutes a link with another portion of the total system. The ultimate 
purpose and mission of this is not important for our present purposes; it 
might be a manned space satellite intended to land on Venus. 

What are the possible kinds of functions this human being can perform? 
And what are the conditions which have to be established for the proper 
functioning to take place; in other words, for the desired kind of input-output 
transformation to occur? It is necessary to ask this last question because, as 
we shall see, the particular kind of human functioning which is going to occur 
at any particular time has to depend upon the provision of conditions to 
sensitize, to trigger, and to control the function, and not simply on the nature 
of the display itself. We have already seen that this is so in contrasting the 
functions of sensing and identifying. A human being is not fixed in his opera- 
tion, but flexible. Accordingly, one must consider several different sources of 
input which determine the kind of input-output transformation that is made 
on any given occasion. Besides the display itself, we have mentioned verbal 
instructions as one of these additional conditions. There 
visual and auditory signals; check-lists; 
physical environment. And in addition 
those inputs to the function which come 
ment, but rather from short- and long- 
A complete description of any given h 
account of these memorial sources of in 


are others, such as 
pictures; and general aspects of the 
. it is necessary to take account of 
not directly from the external environ- 
term memories within the individual. 
uman function requires that we take 
put as well as of the external sources. 
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simply vary in a continuous fashion along several dimensions (such as dura- 
tion, speed, and force), without actually requiring different categories of 
neural organization. 

For practical purposes, three different categories of human responding 
are often recognized. There is, first of all, the kind of output which performs a 
required action (like pushing a switch) in a relatively brief time, without 
change in direction during its occurrence. These may be termed unitary 
responses and occur fairly commonly as output modes. Secondly, there are 
responses which require a continuous, uninterrupted movement, like turning a 
crank, but which appear to be relatively self-contained and internally con- 
trolled. These outputs may be called autonomous sequences. They are char- 
acterized by the capability of “reeling themselves off" with very little guidance 
from external stimulation during their occurrence. The operation of cranks, 
handles, and wheels, as well as the use of familiar tools, are common examples 
of this kind of output. Finally, there is responding in a flexible sequence, as 
One does in tracking an irregularly moving target. Such sequences are guided 
and modified in critical ways by input signals from displays and the other parts 
of the human operator's environment. Tracking tasks are of Find as ge 
Occurrence in systems, although the limitations of Speed Y Mec : 
humans responding in such tasks sets some restrictions on t ved re 30 
fixed and flexible gunnery in aircraft, for example, ph a um ges enen 
Way to tracking performed by peer aer E of the rapidity of tracking 
Mov ired in modern aerial combat. 
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able set of clues about the nature of the function with which we are dealing. 
For example, if the pressing of a switch is employed to convey the information 
that a change in physical energy has occurred, we can say that the human 
function of sensing is being utilized. If the pressing of a switch (one out of 
five) is done to provide the information that a particular class of wave form 
is present in input, then we may suspect that the function of identifying is 
being used by the human operator. The fact that the output is unitary or 
sequential, made to a switch or joystick or handwheel, is not of consequence. 
But the way this mode of response is related to the input is what provides us 
with an essential criterion of human functioning. 


The Varieties of Human Functioning 


Now let us return to our fictitious system and see if we can first illustrate 
and then describe the nature of various human functions, beginning with the 
simplest. We shall imagine that a single human operator is seated at a panel 
having several different kinds of display and several different controls. What 
we shall try to do is to state the following conditions for each function we 
come upon, noting in each case that these differences are what determine the 
kind of human function with which we are dealing: 


1. Minimum display requirements 
2. Filtering conditions 

3. Shunting conditions 

4. Memorial inputs (long-term) 
5. Memorial inputs (short-term) 


These five conditions differ for each of the functions we shall describe. A 
succeeding section will give an account of a typical human task for each of 
the functions and an idea of the differentiating features of each. 

Every function may be described, as we have said, as a kind of input- 
output transformation. It is important to note that a full delineation of such a 
transformation requires the statement of all five of these varieties of input; 
it would be quite insufficient to specify only the nature of the physical instru- 
ments or other energy sources that constitute the “display.” Before considering 
the other input conditions, however, we may note that this type of input must 
be specified in terms of minimum display requirements. One cannot expect to 
transform some information which is not there; thus if sensing is the function 
of interest, there must be a physical difference to be sensed, rather than a 
blank screen. A second requirement is the establishment of filtering conditions. 
The individual does not respond to the entire range of potential inputs in his 
environment; in fact, he responds at any given moment to only a few of them. 
If he is required to identify certain shapes on an oscilloscope, the chances are 
that he is at the same time required to ignore other shapes which are con- 
sidered as "noise." Third, as we have already mentioned, there must be à 
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persisting condition of sAunting, so that the individual will make the desired 
kind of transformation. If he is expected to identify patterns (an inter- 
mediate-level transformation) it may be required that he avoid reporting the 
meaning of these patterns (a higher-level transformation). Both filtering and 
shunting conditions are frequently provided by instructions, but these instruc- 
tions establish a continuing state in the human being, often called a set, which 
is what accounts for the persistence of these conditions throughout the per- 
formance of any given task. It may be readily seen, therefore, that such sets 
are carried by a mechanism of short-term memory, or “buffer storage." 

The mechanisms of short-term and long-term memory are presumably 
very different in the human being, just as they are in the computer. Short-term 
memory has a distinctly limited capacity, as many studies have demonstrated 
(see, for example, McGeoch and Irion, 1952). But in the human being it also 
Possesses an enormous flexibility; and as a consequence, the programming of 
shunting and filtering conditions may be changed readily and frequently, sim- 
Ply by providing suitable instructions. Relatively short sequences of action 
may also be held in the short-term memory. Long-term memory, on the other 
hand, has a very considerable capacity, the limits of which have not yet been 
Very well described. Programs of filtering and shunting may be retained in the 
long-term memory for extensive periods of time, if one desires, but changing 
them within the long-term store is not an easy matter. However, the major use 
of the long-term memory can be seen in the retention of internally organized 
TOutines which are variously referred to as *models," "rules," or "action 
Sequences." For example, in order for a change in wave form to be identified 
by the human operator as a "frequency change," there must be an internal 
Model of such a change, stored in long-term memory, with which the input 
from the display can be compared. Furthermore, as we shall see, such a model 
is a highly complex routine, relative to those of a typical machine. 


Sensing 
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Composition, so far as visual displays are concerned, and comparable differ- 
ragag may be sensed when input is fed through other senses, such as hearing 
9r touch, Considerable quantitative information has been collected — 
ese varieties of sensing, particularly for vision and hearing (see — Jy 
As we have said, the function of human sensing is not used with par- 
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ticular frequency in modern systems, largely because machines can be designed 
to do a superior job. However, the specific variety we have used as a 
example, visual acuity, is sometimes utilized to good advantage in such tasks 
as alignment of vernier scales and in aligning instruments such as transits and 
gun sights. It is difficult, though, to think of an example of the use of bright- 
ness or color sensing as a function of a human operator in a system. Similarly, 
although auditory inputs are often employed in systems both as warning sig- 
nals and as conveyors of voice communication, the sensing of slight physical 
differences in sound energy does not often occur as a task for a human 
operator. Although the mechanisms for sensing are activated in the use of 
higher functions, sensing itself has extremely restricted employment in systems. 

A number of studies of sensing thresholds have shown that filtering con- 
ditions have important effects on the exercise of this function; in fact, the 
various constant errors which occur in psychophysical experiments may bc 
considered to reflect the limitations of the filtering mechanism (compare 
Woodworth, 1938). The differences in results obtained in psychophysical 
experiments with trained versus untrained observers are also relevant to thc 
establishment of filtering conditions. Evidently a trained observer does not 
"sense better" because of his previous training, but he does "attend better" 
and thus is able to report extremely small physical differences with a high 
degree of probability. The implication is that certain observational routines 
may be acquired and stored in long-term memory and that these may facilitate 


filtering. Although filtering conditions for the sensory function may readily be 
established by instructions ( 


“As soon as you see a difference in brightness in 
these two patches of light, 


press this switch"), these will not be optimally 
effective in untrained observers. 


What about the other conditions for the sensing function? Shunting con- 


ditions are readily established by instructions (“Report only a difference in 
brightness, but not how bright the patch is, or what color it is, or anything 
else”). As our previous discussion has implied, there are presumably no effects 
of training on the Sensing function; inputs from memory affect the function 


only indirectly, through their influence on the filtering conditions. From the 
point of view of systems design, perhaps the most important aspects of condi- 
tions for sensing are the minimum 


A display requirements expressed in terms of 
physical measures. Because the sensing mechanism is the initial avenue for all 
higher functions (see Figure 2.2) 


; > ; the physical limitations of this function 
sometimes have an important bearing on equipment design (see Chapter 5). 


These limits of sensory functioning are described at greater length in Chapter 4. 
Identifying 
A much more im 


portant and widely used function than sensing is the 
function of identifying. Recall our example in which the human en is 
asked to “report,” by p 
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patterns is being displayed. This kind of function is perhaps the most ubiqui- 
tous in which human beings engage, whether in their daily lives or in the 
context of systems operation. A stock clerk identifies merchandise; a pilot 
identifies panel instruments; an office manager identifies his staff; an equipment 
repairman identifies the parts of equipment with which he deals; a file clerk 
identifies file cards. Learning the names for and locations of the various 
controls of a machine, or the words to be used in verbal communication 
Within a system, are examples of this function. In identifying, the human 
Operator makes a number of different responses to a number of different 
classes of stimulation. The transformation which takes place from input to 
Output performs the specific function of identifying (distinguishing among) 
different input qualities. In more traditional psychological language, this is the 
function of perception. 

A particular case of the identifying function must be recognized, in the 
Situation which requires the identification of only two input classes, as opposed 
to three or more. For example, we may want to require an operator to report 
the presence of a “blip” on the oscilloscope face. But this means that we 
expect him to distinguish a blip (a small bright patch having certain size 
limits) from that large class of other small bright patches, the “noise” in the 
Machine, In such a form, this function is often called detection. Although it 
May be said to have a certain simplicity in this form, since the inputs to be 
distinguished are only two, the function is still basically one of identifying, 
that is to say, of making different responses to different classes of input. In all 
Probability, the conditions for generation of this function are basically the 
Same, whether we are dealing with twenty input classes or only two. 

. Filtering Conditions. In getting the operator to report certain classes of 
Input qualities, one must utilize a set of instructions which prevent the report- 
'ng of other input qualities (since such reporting would contribute "noise" to 
the next input). If we require the identifying of changes in wave amplitude 
and frequency, for example, this may mean that we do not want the reporting 
Of changes in wave brightness, or its position on the scope face, or its "thick- 
Ness” as a line, or any of a number of other qualities it may possess. If the 
input is Morse code, we want the human operator to identify the particular 
Combinations of sound that are used to make letters and words, but not such 
Other stimulus qualities as signal speed or loudness. Instructions, as well = 
Memorial inputs from previous instructions, define the identifications require 3 

Shunting Conditions. Another purpose of instructions is, of course, = 
Shunt out the employment of higher-level functions on the part of the human 
©Perator. The task may be one which requires him to perceive but E to 
Interpret, For example, if he is asked to identify five different wave forms, 
this may imply that he is asked not to report any of several meanings of these 
Wave forms, such as their possible means of generation or their implications 
85 inputs for the next stage of the system. Reporting such meanings may be 
easy or hard for the individual. But if the designed task calls for identification 
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only, then the shunting conditions must also be designed to keep the individual 
functioning at this level, or else his output will be likely to contain errors as 
well as noise. 

Memorial Inputs. Obviously the filtering and shunting conditions of 
such a task are carried as programs (or sets) in the short-term memory, after 
having been established by instructions. The long-term memory, on the other 
hand, provides the “models” of wave forms with which the external inputs are 
compared and which thus make possible the differentiation of output that is 
required in five different responses (for example, pushing five different buttons). 

The nature of these stored models is an interesting and remarkable mat- 
ter. As is well known, the physical nature of the input may vary over a 
relatively wide range without disturbing the identifications which are made. 
This is what is meant, in its broadest sense, by perceptual constancy, and it is 
a property which has not yet been built into a machine, except in the most 
limited sense. As a simple example, it is evident that a human being can 
readily identify the digits 0 1 2 3 4 5 6 7 8 9, whether they are printed, 
stamped, or written, and even when they are subject to wide variations in 
size, stroke width, or other changes in form. Up to the present, at least, there 
must be a close physical resemblance between two digits for a machine to be 
able to “recognize” them as the same; for a man, the physical resemblance 
required varies over a wide range. Because of such models in long-term mem- 
ory, a photograph of an object is an input equivalent to the direct sight of the 
object itself; a chord in the key of F is equivalent to the same chord in the key 
of G; a round knob among a group of square knobs is identified regardless of 
changes in its size. In a more complex setting, a ground radar return may 
make possible the identification of objects such as railroads, bridges, and 
bodies of water as equivalent to those depicted on a printed map. 
hand pawa or models, are chiefly acquired by a process of 
innately determined such a ones n ie miei eroi ond a 
adi conti i ant hose which underlie figure-ground identifications. 

; : ain figures and patterns, emphasized by Gestalt 
psychologists (see for example Woodworth, 1938). But models like those used 
voy Canine ig qp E Pm Seng pte and spoken 
Dub inpr! PONE e a microscope, or filing cards ina catalogue 

performing of the task requiring the identifica- 


tions. The optimal conditions for s i 
uch learning aj ell 
understood even today, ferme dh: 


although experiments on hu i is- 
tory extending back several decades (McGeoch D Din [7x am 
clearest Set of conditions which emerges from work on paired-associate learn- 
ing is that of contrasting right and wrong examples of the model to be 
acquired In successive practice trials, and Providing immediate “knowledge of 
results after each trial. In this way the models to be acquired tecum 
progressively "differentiated" (see Gibson, 1942), as steve y he pro- 
gressive reduction in errors as learning proceeds. i TON 
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Identifications are sometimes put together to form sequences of action, 
rather than being acquired one at a time in any order. Such sequences are 
usually referred to as motor skills, or perceptual-motor skills. In a common 
example, reading out loud from a printed page, the individual must identify 
cach word in the sequence by making a response appropriate to it (that is. by 
Pronouncing the word) and then going on to the next. Of course, an entire 
sequence of this sort may also be committed to memory, in which case the 
input for each identification becomes, not the printed word itself, but the 
sound of the previous word uttered by the speaker. Similarly, in a skill such as 
bicycle riding the input for each successive identification (such as pushing 
downward on the left pedal) is a complex which includes both the currently 
Present external stimulus situation and the proprioceptive feedback from the 
movement just previously made (extending the right leg). When the human 
individual is required in a systems context to track a continuously moving 
display element, such as a spot on an oscilloscope, he is similarly required to 
use the identifying function in response to an input that includes both the 
Current position of the spot and the feedback from his previous movement, in 


4 continuous action sequence. 


Interpreting 


In many system instances, the most important function performed by 
the human being is interpreting. Here the individual makes outputs which, in 
effect, place inputs into categories whose basis is their effects rather than their 
appearances, In other words, interpreting is a matter of identifying the mean- 
mg of inputs. Suppose that our human operator, seated at his console con- 
taining oscilloscope and response buttons, is given the task of reporting 
Whether a blip on his scope represents à friendly or a hostile aircraft. He 
Cannot, of course, decide on the basis of its appearance, since this is not 
distinguishable from that of other blips. Instead, he may have to decide 

s which form a part of the memorial input 


Primarily on the basis of certain rule ; t 
in this case—the blip is approaching from the northeast, flying at a certain 
altitude and with a certain speed, following no previously reported "track, 


and so on _ ae 
The contrast between identifying and interpreting is well exemplified in 


the task of aerial photo-interpretation. The initial part chalet sus = 
Which the human being identifies ciel un a ped ae EN ms 
akes. But in interpreting the photograph, 2e takes anothe dh «e 
'nterprets i “combat aircraft,” or a building as a “power plant, 

or i Nate - E ec It should be noted that he is not able x do = 
9^ the basis of the appearance of these objects (a water tank: may. : in - 
tinguishable in appearance from a fuel tank) but only by making in A 

about the expected efjects of the actions they lead him (or others) to take. E 
Other words, he begins to work with “if-then” relationships, which can be 
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approximately illustrated by the statement "If it is a power plant, then I 
should see transmission lines." Psychologists have given this kind of behavior 
many names, including “expectancy,” “hypothesis-using,” and “thinking.” In 
interpreting, the human being identifies inputs in terms of their expected 
effects. Interpreting thus involves a choice among “courses of action" avail- 
able through memorial inputs. 

The minimal display conditions required for interpreting are determined 
by the fact that the function of identifying must precede the interpretation 
itself. In the case of the task of interpreting a blip on an oscilloscope, for 
example, the human operator must have been able to identify the northeast 
approach, the altitude, the speed, and the type of track followed in order to be 
able to make the interpretation required. In the photo-interpretation task, he 
must be able to identify transmission lines, transformers, and a number of 
other objects in order to perform the interpreting adequately. Unless the 
required objects are displayed, and unless they can be identified by the human 
Observer, correct interpreting cannot be accomplished. 

Shunting and Filtering Conditions. At this level of human functioning. 
shunting conditions are not required. The channels are open, and the human 
operator is being asked to function in a manner which he seems to prefer and 
at which he often excels. 

Filtering conditions, however, are quite as essential with this mode of 
functioning as with others. Instructions are used to “set the problem," or, in 
other words, to tell the human operator what to look for. If he has the task 
of reporting enemy aircraft, he must be instructed, in effect, not to report such 
things as the volume of traffic on local airways, or the fact that a scheduled 
commercial airliner is ten minutes late, or a whole host of other interpretations 
which he is perfectly capable of making. Sometimes he may have time to make 
such interpretations, to attend to such inputs; but on those crucial occasions 
when the load is heavy, filtering will be essential to the prevention of errors. 

j Ordinarily, as we have said, filtering conditions are established by means 
of Instructions which activate sets carried in short-term memory. There are 
occasions, however, on which system functioning may depend upon the dis- 
covery of suitable filtering conditions by the human operator himself. For 
example, suppose the task to be performed were that of interpreting an aerial 
photograph of enemy territory for “military significance.” In such a case, it is 
pics na posible or at least not desirable, to specify by instructions “what 
i ? own—he will adopt his own sets by giving himself 
instructions. He may t i 1 s 

é „may try several of these self-instructions in a row before 
ith or which make it possible for him to perform the task successfully 
© his satisfaction. Such self-instructions appear to be equivalent to what à 


number of psychological investi 
igators call "strategies" ex 
Bruner, Goodnow, and Austin, i meas nang t 


1956). System develo 
: pment procedures are 
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human beings in discovering their own best filtering conditions. We shall have 
Occasion to return to this point in a later discussion of "systems learning." 

Memorial Inputs. Short-term memory provides, as we have said, the 
sets which determine filtering. There may also be brief sequential routines, 
actually a part of the filtering mechanism, which determine certain sequences 
of behavior. For example, such a routine might read: "First, check the alti- 
tude, next the range, and finally the speed." Such routines must of course 
contain a limited number of steps if they are to remain within the limitations 
of the short-term memory. In any case, it can be seen that their function is to 
provide for the filtering of inputs in a certain order. 

One kind of input from long-term memory is a familiar one: the models 
Which make identification possible. The individual, it is often said, must 
Perceive the situation before he can interpret it. We have already emphasized 
this point. If the human operator is being asked to report "enemy attack," he 
must be able to identify such things as altitude, range, speed, and nature of 
track. This means he must possess a mechanism which compares the actual 
inputs with models available in his long-term memory. Similarly, the photo- 
interpreter must be able to identify, on the basis of stored models, power 
transmission lines, transformers, and many other objects, in order to make 


the interpretation, “power plant.” 


The second type of stored input is n s of th 
of which objects, regardless of appearance, are classified into categories of 


“expected effects.” They are the action categories which are so essential to 
Interpreting. In landing a large aircraft, the pilot perceives instrument indica- 
tions of a number of aspects of the plane’s performance, or even more 
Immediately, in terms of the courses of action available to the pilot himself. 
Thus an overly high altitude may “mean” the alternative courses of action of 
Nosing down or making another landing approach. The rules that define these 
courses of action are available to the trained pilot from his long-term memory. 
These rules can be expressed as verbal statements; whether they are stored in 
this form or some other is a matter psychologists argue about, but which need 
Not concern us here. Similarly, the photo-interpreter who has identified a 
“pile” has in his memory a whole set of rules which determine a number of 
Courses of action, such as “Look for vehicle tracks" or "Look for nearby 
Chimneys,” which enter into the process by means of which he arrives at the 
Interpretation “coal pile" that is the output response. It may be noted, too, 
that these stored rules are not necessarily of an all-or-none sort; rather, they 
frequently represent probabilities and have been studied as such by a number 


9f investigat Gagné, 1959). 
n UR Decision Making. It is not at all 


Interpreting in Sequence: Dynamic 
Uncommon for interpreting tasks to occur in sequential form. The step-by-step 
Nature of problem solving has frequently been emphasized (see for example 


Johnson, 1955), as well as the fact that the information available for each 
ater step is likely to be contingent on the consequences of earlier ones. In a 


ew. It consists of the rules by means 
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task of equipment troubleshooting, for example, the initial interpretation may 
lead the individual to check an input voltage; his next step is an hans 
of symptoms which include the initial information plus this voltage — 
At each step of the way, the input to the human problem solver is changing. 
because of his own actions, and continues to do so until he is able to make 
the final interpretation that locates the malfunctioning part. Acrial photo- 
interpretation also frequently proceeds in this sequential fashion (Gagne, 1954). 

Obviously, this kind of dynamic decision making requires successive 
reprogramming, at each step of the way, of the filtering conditions in short- 
term memory. Thus at each new stage the individual must provide his own 
instructions to establish the filtering set which tells him “what to look for.” 
This kind of flexible reprogramming constitutes one of the most striking 
characteristics of human functioning, and one which distinguishes it markedly 
from that of most machines. In performing in this manner, the individual must 
have available in his long-term memory the variety of “filtering rules” neces- 
sary and relevant to the problem to be solved. Choosing among them is the 
major task accomplished by interpretation. The rules themselves then come 
into play in determining the particular filtering set employed in the next stage 
of the problem; they are self-instructions, or programs. 

A more complex form of dynamic decision making occurs when the 
feedback loop from the preceding action step involves interacting with another 
person. In such circumstances the input situation for cach step in the sequence 
becomes highly variable and unpredictable, since it depends not solely on the 
effects of the individual's own output, but also on that of the second indi- 
vidual. Suppose that two air-traffic controllers must share the task of guiding 
traffic into an airfield, having as a primary display two PPI (Plan Position 
Indicator) radarscopes displaying the same information. This is a "load- 
Sharing" task, and it is obvious that each decision made by individual A will 
result in an action that changes the input for individual B. In periods of heavy 
traffic, such a system could presumably not be operated without some exter- 
nally imposed filtering instructions concerning the sector, range circle, Or 


destination to be covered by each controller (Kidd and Hooper, 1959). 
However, even when this has been done, the task is 
interaction is Tequired between the individuals 
situation for decision making is determin 


the other person. The possibility of flexible programming is obviously a neces- 
sary condition for successful operation of this type of system. 


Inventing. We have seen that the high adaptability of human beings is 
exhibited in their capability for frequent reprogramming of the filtering (as 
Well as shunting) conditions applicable to the 
50 another, and even mor 
an behavior is demonstr 
rules. It is reasonable to suppose that 
matter of using old rules (ideas) 


one in which frequent 
. in the sense that each new 
ed in part by the preceding action of 


puts. Th € striking, way in which the 
flexibility of hum ated. This is in the invention of new 
"having a new idea" is fundamentally a 
in utterly new situations. The history of 
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Science contains many examples in which a principle from one sphere of 
knowledge is transferred by analogy to another sphere, with the result that 
the entire outlook on the second problem is changed, the problem is “rede- 
fined," and a great upsurge of novel theory results. Invention also occurs with 
Some frequency in other areas of human activity as well, and its occurrence 
as a requirement in certain systems cannot be overlooked. A successful leader 
is often an inventive leader, in the sense that he achieves new solutions to 
problems of social interaction and group productiveness. Military tactics and 
Strategy also require new solutions for new situations if they are to remain 
Successful. 

Inventing occurs when the individual makes an interpretation of inputs 
in accordance with a set of rules which he has not previously applied to the 
Situation or to similar situations. Thus, for the individual, every "original" 
Solution of a problem is, in a sense, an invention. Whether or not it is an 
invention in a broader social sense of course depends upon whether other 
People have previously made the same interpretation. Much of the evidence 
about inventing is anecdotal (see for example Platt and Baker, 1931 Js It is 
interesting and suggestive, but of limited scientific usefulness. “Brainstorming” 
(Taylor, Berry, and Block, 1958) is a technique which was designed to 
Stimulate invention in group situations, and its encouragement of “wild ideas” 
is a feature which suggests a possibly new experimental approach to the topic. 


Review of Human Functions 


It will be useful now to summarize what we have said about the three 
basic types of human functioning, which we can do by reference to Figure 2.4. 
In addition we need to discuss some of the implications of this description of 
functions, and particularly to see where they lead us in relation to the develop- 
Ment of systems. g " " 

Human functions are varieties of transformations which the human being, 
Considered as a system component, performs upon inputs to produce outputs. 
They have been identified and described by considering a typical unitary 
n being who is presented with an equipment 


SUbsystey ced ata * 
System, composed of a huma 
E buttons and knobs as 


display and a set of controls that include manipulable oq à 
Well as a microphone for verbal communication. In such a seng, it can be 
Seen that the input from the display (which itself is a machine output) is 
transformed by the human being with the use of controls into an output 
(which in turn becomes an input to the next element of the system). Thus the 
transmission of information is, in general terms, the use which human beings 


fulfill in systems. 
When we analyze the n 
We find they are composed of several p : ; 
'Splay itself (shown here as an oscilloscope wave form), SHIT 
ich establish sets in short-term memory (similar to "bufler storage ). These 


ature of inputs to the human being in their totality, 
arts, as Figure 2.4 shows. Besides the 
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The basic varieties of human functioning. 


sets determine the shunting conditions 


which the individual operates in the situation. In addition, they establish 
filtering conditions, which have the very important role of selecting inputs— 
that is, of determining what the human operator attends to. Brief sequential 
routines of observation may also be held in the Short-term memory. Finally. 
there are inputs from long-term memory, the most important of which are the 
models, which make possible comparisons with display inputs to produce 
identification, and the rules, which Tepresent courses of action to be compared 


» and thus the level of functioning at 
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in the function of interpreting. Sequences of action are also stored in long-term 
memory, forming the basis of perceptual-motor skills. 

Sensing is a function that requires a difference (or change) in physical 
energy as a minimal display condition. Filtering conditions establish attention 
to a particular difference, while shunting conditions act to prevent the report- 
ing of “things” and “meanings.” Filtering conditions are often partly a matter 
of observational routines stored in long-term memory, as is shown by the 
superior performances of “trained observers.” While many sensing functions 
of human beings are very acute (see Geldard, 1953) they are nevertheless 
readily excelled by those of machines; as a consequence, this kind of human 
function is not widely employed in systems. However, sensing obviously must 
be accomplished in every human activity, and in this manner it sets the limits 
to all functioning. "" mu 

Identifying represents the next level of functioning, as shown in Fig- 
ure 2.4. When suitably defined, this function is equivalent to “perceiving.” 
Minimal display requirements are the presentation of two or more aspects of 
Stimulation to be reported differentially by an equal number of responses. In 
the simplest case the presence or absence of an input feature (such as a 
"blip") is to be identified against a background of “noise”; and this variety 
of identifying is usually called detection. In identifying, instructions play their 
Usual role as inputs for the establishment of shunting and filtering conditions. 
The most important memorial inputs ( from long-term memory) are the 
Models, which provide standards against which display inputs are compared 
and thus identified. " ! 

Interpreting is the highest level of human functioning and takes place 
When shunting conditions for lower functions are removed. The individual 
makes Output responses which classify inputs in terms of their effects, rather 
than in terms of their appearance (as in identifying). To do this he must use 
Tules that are stored in long-term memory and that represent pem 
Courses of action. A particular variety of interpreting, called inventing, occurs 
When the individual uses rules which are novel (for him) ac respect to the 
"puts he is responding to. Actually, there are many specific varieties es 
terpreting (like numerical ecnputing aa eub T 
TOubleshooting) in systems, and we shall be £ " bid - 

One characteristic of interpreting deserves a special mention, E 
also be consi ore fully in a later section. AS Figure 2.4 shows, a tota 
act of Diis peste be ck exercise of the preceding d nens 
and identifying. However, it is well aie! oe [inse ifia nel These 
able to use inputs which simply represent the a id : ivi 
are symbolic i ich, in a sense, simulate the outputs of the ident ying 
"aper ae ar nar normally occur. For example, if the à m 
MVolves as inputs the digits 1, 2, and 5, it is possible to represent these 

© words one, two, and five. Interpreting 


can then proceed without error, 
Provided that the rules used have been learned in such a way that the digits 
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and the words are equivalent, as is often the case. This provision, however, 
must be satisfied: one would have to make sure, for example, that words such 
as un, deux, and cinq were identified in an equivalent manner to 1, 2, and 5 
before the former could be successfully used as "substitute" inputs for an 
interpreting task. This feature of interpreting has important implications for 
training as well as for the use of simulation in systems. 

The occurrence of interpretations in sequence, in response to input events 
which are contingent upon the individual's output in the previous step, consti- 
tutes a variety of interpretation called dynamic decision making. It involves 
frequent changes in the filtering sets, a mechanism which appears analogous 
to reprogramming in the sense that new self-instructions must be provided by 
the individual for each stage of action, consistent with the new input situation 
occasioned by his just-previous response. When another link is added to this 
feedback chain in the form of another individual, the input for each decision 
becomes accordingly more complex and more difficult to predict. 


Some Implications 


What have we accomplished by viewing the individual as a system com- 
ponent having the purpose of transforming inputs to outputs? Have we added 
anything that would not be available by a study of psychology in more tra- 
ditional terms? 

The answer appears to be yes and no. It is clear, for example, that the 
psychologist is fundamentally dealing with the same set of phenomena. Inputs 
are called stimuli, outputs are called responses by the psychologist. In using 
one as the independent variable, the other as the dependent, his purpose is in 
fact to make inferences about the mechanisms in between. In so doing, he long 
ago established the categories of sensing, perceiving, and thinking (all of 
which are sometimes given various other names) as forms of behavior related 
to qualitatively distinct sets of experimental operations. Furthermore, he has 
for an equally long time investigated the properties of memory 


x as well as 
immediate memory. The selective or filtering function of attention, acting 
through the mediation of sets, 


alb L ‘ has often been a focus of interest for the 
psyc ologist. In this sense it cannot be said that we have made new distinc- 
tions, nor identified new phenomena. 
i5 inris s Be that the mechanisms we have described, as well as 
ncuons they define, represent a basic set of i a i 
set of infer ed 
2 HUNE sien nferences that one is forc 


he observes the variet i ible i i 
y of behavior possible in cing. 
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à "hypothetical constructs," and ot go 
very far to explain behavior in the sense that theory is erui, ig de 
language proposed by Northrop [1947], they are "concepts by intuition" 
rather than "concepts by postulation"), Accordingly, they are not intended 
to account for the why and how of sensing, perceiving, and thinking, but only 
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to distinguish these as human activities requiring different sets of manipu- 
lations of the system environment. 

These functions are considered to represent the initial, organized way 
which must be used to deal with human behavior in the framework of systems. 
They appear to be quite compatible with the functions used to describe 
machines, so long as the latter are expressed in terms of “data transforma- 
tion.” In addition, the conditions needed for their use are quite analogous to 
such machine mechanisms as filtering, programming, and shunting. They 
should therefore provide a language which can be related directly to the other 
events and operations that occur in systems. 

In thinking about man in a systems framework, it is essential that one 
deal with functions rather than with the unitary events that compose them. 
Stimuli and responses are perhaps necessary to psychological analyses, but 
they contribute little, if anything, to precision of thinking about human func- 
tions, over and above the concepts of input and output. As for S-R “relation- 
Ships," our analysis leads us to believe that these can be no simpler than the 
mechanisms we have described, and therefore offer no advantage in this 
Tespect. A number of implications flow from the consideration of human func- 
tions as we have described them, important to system development and per- 
haps also for psychological science. Let us take a brief look at some of these. 

Selection and Training. The men who take part in the operation of a 
System have to be selected and trained (see Chapters 7 and 9). They may be 
Selected because they already possess the capabilities for performing in a 
System; or more typically, because they are capable of rapidly acquiring, by 
Means of training, the specific capabilities needed. The human functions as 
described here have immediate implications for selection and training, and 
these may be seen to be generally applicable to all kinds of jobs that occur 
in systems, . . 

If the task to be performed involves sensing, selection must screen out 
those who fall below a given standard applicable to the particular sense mode 
Concerned, whether it be a matter of visual acuity, brightness discrimination, 
uditory-frequency discrimination, or whatever. Training can be designed to 
establish stable sets for filtering (that is, it can produce the € a 
Server”), but beyond this nothing can be achieved. If identifying is p fo 
the main function of training is to establish the models required for this 
function, whether these be wave forms, objects in aerial Shans? eighties 
Words, or symbols used to provide computer inputs. A ea i cies 
and additional purpose may be the establishment of p cn rd 
35 are used in perceptual-motor skills. Selection may be employed to p 


those individuals who can acquire an 
Tapidly. If the task involves interpreting, 
function of identifying can be performed 
en established by training. Beyond this, 


one must first insure that the prior 
and that the required models have 
the function of training is to insure 
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the acquisition of the rules needed to make the desired transformations, as well 
as the programming rules used in the establishment of filtering sets. Again, 
selection may be used to find those individuals who can most readily accom- 
plish the storing of these rules. 

Job Aids and Manuals. There are also some obvious implications of our 
analysis of human functions for the development of printed aids and manuals 
used by the human operator in the course of system operation (see Chap- 
ter 8). First of all, we have found it necessary to recognize at the outsct that 
inputs to the human components are not limited solely to the display of 
equipment. A highly important input source is the instructions provided to the 
operator. While these may on occasion be oral, they are most frequently 
printed, occurring as a portion of job manuals, as standard operating proce- 
dures, as check lists, and in other forms. One of the main purposes of such 
instructions is to bring about reprogramming—in other words, to accomplish 
the necessary changes in sets for filtering and shunting which are needed as 
the individual proceeds from one mode of operation to another. There is a 
second major purpose of printed materials, in addition to this instructional 
one. Such materials are needed as a means of maintaining retention of the 
models and sequential routines in the long-term memory, particularly when 
these are complex. For example, a lengthy procedure for putting a piece of 
equipment into operation may be expected to involve a stored routine which 
profits from frequent "relearning" trials undertaken with a check list. Similar 
reasoning applies to a complex model such as that needed in equipment 
trouble shooting; retention of the model may require frequent reference to 2 
“functional diagram" provided in a job manual. 

The Composition of System Tasks. A later chapter (Chapter 6) will 
consider in some detail how the structure and purpose of a system defines the 
tasks to be performed by human operators and how these tasks are combined 
to form jobs. As we shall see, the tasks generated by a system are simply 
small parts of larger operations and are describable in common-sense terms 


without resort to technical language. Thus a task may be a suboperation such 
as "plotting tracks of aircraft," “locating targets on a map,” “aligning set x 
prior to operation.” It is clear, however, that the description of a task does 
not in itself provide an indication of the human function involved in it (sce 
Miller, 1953). Locating targets on a map could be a task requiring fairly 
simple identifications, or relatively complex interpreting, depending on the 
kind of input-output transformation required. Thus dig: nal of tasks which 
make up a given job may vary widely in terms of the human functions they 
involve; and a given task may partake of more than one function 
sc sd a 
nize » provide only the starting point for the 
problem of utilizing human capabilities in systems. Proceedi frm His obi t 
one must make an analysis of each ta ee durs prd aA Ao 
functions that are Bend together enn n 
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of them. This is the kind of information required for the specification of 
selection and training requirements, as well as for the design of job aids, as we 
have already mentioned. Thus it may be realized that the identification of 
human functions in system operations provides the key to many of the steps 
that have to be taken in developing the system as a whole, and particularly 
the human components of the system. It is these human functions which will 
basically determine how the human beings are to be selected, what is required 
for their training and retraining, how they are to be instructed during system 
operations, and what standards can be set for their performance. 


Limitations in Human Functioning 


Our discussion of human functioning has revealed a number of limi- 
tations of human functioning, some of which we have mentioned rather 
casually as we proceeded. It is worthwhile, however, to take a direct look at 
this problem. What are the limitations of the human functions we have 
described, and how do these compare with those of present-day machines? 
This is a question of particular importance in the design stage of system 
development, when the designer may be faced with the necessity of comparing 
the consequences of substituting a machine for a man or vice versa. 

What are the limitations of human functions, relative to those of ma- 
chines? Several authors (see for example Chapanis, 1960) have contributed 
lists of advantages and disadvantages of men and machines, expressed in such 
terms as “flexibility of programming,” “size of channel capacity,” “perceiving 
organized patterns,” and so on. Is it possible to express these ideas, perhaps in 
more highly organized fashion, using the functions and conditions we have 


described? Table 2.1 makes this attempt. 


Table 2.1 
ations of Machine and Human Functions, 


Some Comparisons of the Limit 


in Relative Terms 


Limitations 


Conditions 
for function Human Machine 
SENSING 
Range extends far beyond 


ain ranges of energy 


Display Limited to cert 
change affecting human senses human senses (X-rays, 
infrared, etc.) 
Sensitivity: very good Sensitivity: excellent 

Filtering Easy to reprogram Difficult to reprogram 

IDENTIFYING 
Display Can be varied over relatively wide Can be varied only in very 
ıl dimensions narrow range of physical 


range of physici ; ; 
dimensions 
Channel capacity: small Channel capacity: large 
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Table 2.1 (continued) 


Human Functions in Systems 


Some Comparisons of the Limitations of Machine and Human Functions, 


in Relative Terms 


Conditions 
for function 


Limitations 


Human 


Machine 


IDENTIFYING (cont.) 


Filtering Easy to reprogram Difficult to reprogram 
Memory Limits to complexity of models Potential limits of complex- 
probably fairly high, but not ity very high 
precisely known 
Limits to length of sequential rou- Potential limits of length of 
tines fairly high, but time-con- routines very high 
suming to train 
INTERPRETING 
Display Same as Identifying Same as Identifying 
Filtering Easy to reprogram Difficult to reprogram 
Highly flexible. i.e.. adaptable Relatively inflexible 
May be reprogrammed by self- 
instruction following input 
changes contingent on previous 
response (dynamic decision 
making) 
Shunting Can be readily reprogrammed to Difficult to reprogram 
lower levels of functioning 
Memory Limitation to rule storage not Limits of rule storage ("log- 


known 
Speed of reinstatement of rule 
sequences relatively low (as in 
computing) 

Use of novel rules 
(inventing) 


ic") quite high 
Speed of using rule sequences 
high (e.g.. computing) 


possible Limited in use of novel rules 


As the table indicates, limitations may be described and understood in 
accordance with the input conditions to which they relate, whether these be 
characteristics of the display, of the filtering and shunting conditions of short- 
term memory, or of the long-term memorial inputs. When each of these is 
spelled out, the nature of man versus machine limitations becomes rather 
clear, and it is possible also to see what future possibilities are likely to develop 
in machines yet to be designed. 

For the sensing function, the limitations of man are quite apparent. The 
physical-energy changes he can sense are limited to those which affect his 
receptors of vision, hearing, touch, smell, taste, and so on (see Chapter 4). In 
contrast, machines are able to sense changes beyond these limits, in the ultra- 
violet and infrared regions of the electromagnetic spectrum, above 200,000 
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cycles in the mechanical spectrum, and so on. Within the limitations of his 
senses, a man’s sensitivity is quite good, since it is known that he can sense 
very small differences (compare Geldard, 1953). But the number of changes 
he can respond to at one time is relatively small—that is, his channel capacity 
is limited. Beyond this, it will be realized that reprogramming man's filtering 
conditions is a matter readily accomplished by instructions; in a machine, this 
would be expensive or time-consuming or both. 

Identifying is a function at which man currently excels the machine. 
Primarily, this is because the stimuli to be identified can vary over a wide 
range in their physical dimensions, as well as in the context in which they 
are imbedded, without producing errors in identification. In other words, the 
models man can learn and store are extremely "flexible." Machines have not 
yet been designed which can perceive inputs of such variability in a reliable 
fashion. Instead, the routines that function as models for the machine must 
have relatively fixed patterns. When these routines can be unvarying, but are 
composed of long sequences (as, for example, in certain types of tracking), a 
machine can readily outperform a man. Long sequential routines require fairly 
lengthy periods of practice on the part of a human being. 

A number of varieties of interpreting can be performed better by 
machines than by human beings. Complex and lengthy sequences of rules can 
readily be designed into a machine, and tasks such as computing can be 
performed in a fraction of the time required by a man. Computers can be 
designed also to include probabilistic rules, and such interpretation can often 
better be done by a machine than by human beings. The filtering of inputs, 
however, can be easily reprogrammed in man, whereas this is difficult in a 
machine. In addition, it has been found desirable in a number of instances to 
permit human beings to develop their own filtering rules, and this sort of 
"adaptation" may be shown to improve man's performance. Can machines 
using rules? Perhaps they will be able to someday, 


invent new ways of YU WIE. Dus 
his function is distinctly limited. The 


although as yet their performance in t i ; 
kind of task at which man clearly excels, compared with present-day machines, 


is one which requires interpreting of inputs which require complex identifica- 
tions in the first place. Aerial photo-interpretation, for example, is such a task. 
Perhaps a machine could be designed to follow the rules required in deciding 
that a picture contains an “aluminum plant”; but a human being would first 


have to identify the objects in the photograph which enter into these rules. 
ons are used together in a man, we have a task which 


an one, and which can on occasion use man’s 
capabilities to the limit. Another kind of human function which reveals man’s 
remarkable capabilities is that of dynamic decision making, in which input 
changes at each step of an interpretation sequence produce reprogramming 
of the individual's filtering sets. The facility and rapidity with which such 
changes can be brought about stand in marked contrast to the capacities of 


à machine. 


When these functi 
appears to be peculiarly a hum 
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z HUMAN FUNCTIONING IN SYSTEM TASKS 


We have already had occasion to note that tasks, forming as they do the 
parts of system operations, do not hold a one-to-one correspondence with 
human functions. There are tasks which utilize a single human function in 
their performance, and there are tasks which utilize several functions. Man is 
a relatively versatile "package." It is not surprising, therefore, that he is used 
to carrying out a great variety of tasks, which are typically designed to “fit in" 
with the functions of machines, rather than being designed specifically to 
conform to the conditions of the separate human functions. 

There appear to be certain typical kinds of task situations in which man 
is used as a part of a subsystem, and these have come to be called by special 
names. It is doubtful that these tasks have any more general significance than 
that they occur frequently in many different systems. But each of them reflects 
the employment of human capabilities in a manner which exploits one or more 
of the advantageous features of human functioning. Let us examine a few of 
these here, in order to show clearly the involvement of human functions as we 
have described them. In doing this, we return to our example of the human 


operator seated at a console, the two components forming a link in a larger 
set of system operations. 


Coding 


There are many varieties of coding tasks in systems. Characteristically, 
the input comprises stimulus patterns which must be identified in terms of 
outputs designed to be employed as inputs to the next link in the system. 
Suppose, for example, that the operator must receive messages through head- 
phones in Morse code and transform these messages into typed English 
sentences for transmittal to the next system component (which in this case is 
another human being). Clearly these activities utilize the function of identifi- 
cation. Filtering sets must be established by instructions which insure that 
the individual will attend only to certain kinds of auditory sounds received 
through the headphones and ignore the noise. The individual must utilize 
some stored models which represent the pattern of dots and dashes he hears 
in order to identify them as letters and words. There is also a second stage to 
this task, which also has the characteristics of identifying: the words must be 
transformed into an output which involves the use of a typewriter. There is 
thus a motor skill required, and the sequential routines involved in typing must 
also be available to the individual through his long-term memory 

If we look a little more closely at this total task, we can de that the 
first stage of identifying (transforming Morse code into English prose) can be 
more efficiently accomplished by interpreting. In fact, this is what usually 
happens if we give an individual enough practice at receiving code (as Bryan 
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and Harter [1899] suggested many years ago). As practice continues, the 
experienced code receiver begins to be able to anticipate whole sequences of 
dots and dashes, because he is interpreting the message in accordance with 
well-practiced “rules” of English prose. This method turns out to be more 
efficient for several reasons. The human message receiver can perform accu- 
rately even when subjected to rather extreme distractions; he can receive even 
when the noise level is very high; and he can receive when there are gaps in 
the message. These facts have some particular significance for the system 
design decision as to how the human operator can best be employed in this 
coding task. If the Morse code can be depended upon to provide a relatively 
constant physical input, a machine can readily be used to translate these fixed 
patterns into English letters and words. However, if reception is going to be 
characterized by frequent noise and lapses, a human being may be needed to 
perform both the identifying and the interpreting required for producing 
reliable (and meaningful) English prose. y 

The second stage of this task, the typing, presents a somewhat different 
picture. The input in this case will be meaningful English sentences supplied 
by the individual to himself. He must now identify the letters and spaces 
which make up these sentences in terms of responses made to the keyboard 
of the typewriter. In order to do this, he must have acquired the sequential 
routines that constitute the basis for this motor skill. This, of course, is well 
within his capabilities; after sufficient practice a high degree of typing pro- 
ficiency can be attained. Thus, if a human individual has been chosen to do 
the interpreting task of the first stage, he can readily be used to ipe de 
identifying (motor skill) function required at the second stage. s enueia 
decision has therefore already been made, and can be seen to Mut a 
Wide variety of coding tasks. When inputs can have a fixed pep : 
Outputs, a machine can perform the entire coding operation very well * Ee | 
In contrast, when inputs must be identified which are variable in d p aam 
dimensions; or when they must be interpreted, because Miis ena ae 
ing have to be made to fill gaps; then the task is one at which a human 


being excels. 


Another Example of Coding. Let us change the situation a little, so 


that we can look at another coding task. Suppose we set am Poona 
Subsystem the task of reporting “deviant courses of aircraft sra salia 
tracked by radar and displayed as a track 9 Kader iis  apcdavisfd" from 
Want the subsystem to do is to identify as deviating or n Sor 
Preplanned courses the tracks of a number of aircraft shown Du. me 
Obviously there are several ways of going about this. One may g 


1t, for example, by requiring the human being to oe T pre 
: i by punching them in a 

Of speed, heading, and altitude to a computer, * ning : 

buttons The oe then compares these variables with a Se 

the preplanned course stored in its memory. When the task is designed in this 


Manner, we are requiring the use of the identifying function of the human 
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routines be particularly systematic (as is true, for example, in airplane "spot- 
ting"), they may be practiced and stored in long-term memory. In any case, 
scanning and search frequently are preliminary to, and coupled with, detection. 

Scanning and search routines as a prelude to detection have often been 
taught successfully to military personnel who must carry out missions in the 
dark (compare Sharp, Gordon, and Reuder, 1952). In this case an initial act 
of sensing some change in visual stimulation (a movement in the dark) puts 
into operation a systematic routine of observing “out of the corner of the eye” 
in order to bring to bear on the situation the more sensitive extrafoveal 
receptors. Such routines must be deliberately trained, because they are directly 
opposite to those used in daytime vision, when the “natural” thing to do is to 
look at the object foveally. The function ultimately employed is again one of 
detection, in which the individual attempts to identify “a moving thing” or 
perhaps “a moving enemy soldier” against a background of nonmoving shapes. 


Monitoring 


Another relatively common kind of task that involves the function of 
detection is called monitoring. A monitoring task is one that requires detection 
of events which occur at relatively rare, randomly distributed intervals over a 
lengthy period of time. The events may be the appearance on the display of 
an object which must be reported, or they may be out-of-tolerance conditions 
of the equipment. The human individual may be required to detect one or a 
number of different events, when and if they occur, during the same time 
period. 

The basic function utilized in monitoring tasks is detection, and this 
means that the individual must have available the models required. If he is io 
report the appearance of a blip on an oscilloscope, he must have the model 
which distinguishes the blip from noise. Or, if he must detect out-of-tolerance 
readings on twelve gauges, he must have the model for each of these. How- 
ever, since the events he is looking for are infrequent and sporadic, a monitor- 
ing task is not easy for a human being to perform with high efficiency over 
periods of time greater than half an hour or so. Scanning and search routines 
become excessively tiresome and subject to inhibition. Attention is said to 
wander; that is, the sets for filtering which are held in short term memory tend 
to fade away, and the inputs to be detected are missed (compare Deese, 1957). 

There are two general ways of effectively designing a monitoring task 
assigned to a human operator. One is to establish the necessary models in 
long-term memory and to depend upon a strong alerting stimulus (like a loud 
noise or bright light) to take the place of filtering conditions. This method is 
often used with danger signals. The maintenance of filtering conditions within 
the individual becomes unnecessary with this method, since the warning signal 
is strong enough to capture the attention and to block the reception of all 
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other inputs for a time. The second way is to provide for the occurrence of 
events at frequent enough intervals so that the filtering sets are in a sense 
"reinforced" and thus maintained (compare Garvey, Taylor, and Newlin, 
1959). A human operator who must watch for the appearance of a blip 
representing an aircraft in an isolated portion of the earth may be looking for 
something which seldom happens. A satisfactory way to maintain the filtering 
conditions necessary for such detection is to increase the frequency of the 
events—that is, the blips. This can be done by deliberately designing the 
machine to “help out" the man, by including a program which simulates the 
occurrence of blips (representing aircraft) at relatively frequent intervals. By 
thus insuring the maintenance of filtering sets, the efficiency of the human 
Operator can be substantially raised. 

Actually, as the work of Mackworth (1958) and others has shown, 
Monitoring is not the kind of task which human beings perform well, except 
under the conditions described in the previous paragraph. The equipment 
designer needs to consider the instances in which the machine can be built to 
Perform this function, which is the case when the events to be detected can be 
reflected as changes in single physical variables. In contrast, when these events 
Partake of the perceiving of forms and patterns varying widely in their physical 
dimensions, the detection may have to be done by a man. In such cases some 
Provision must be made for insuring the maintenance of sets for filtering. 


Tracking 


The tracking of a continuously moving target is a kind of task which has 
Occurred with some frequency in military systems. This is the task which 
characterizes antiaircraft gunnery, for example, and, in a somewhat more 
Complex form, aerial gunnery. The function utilized in tracking is obviously 
identification, and the models involved pertain to the “sighting picture” pro- 
vided by the reticle, or gunsight, which displays the target in reference to the 
Positioning of the gun, usually represented as the intersection of cross hairs. 
The directions of movement of the control (the output) reflect the identifica- 
tion of a variety of different positions of the target in the field of the gunsight. 
The task is one in which successive pictures are reacted to (identified) one 
after another by movements which blend into each other and form a more-or- 


less continuous movement. 

In this kind of tracking task, the se d is nc 
Only variable but unpredictable from one moment to the next. This is in 
Contrast with a motor skill like that involved in rendering a particular com- 
Position on a piano, in which the sequence of movements is entirely deter- 
mined by the sequence of notes. Consequently, the models of primary impor- 
tance to the tracking task are those which represent relationships between 
target position at any given moment and direction and speed of movement of 


quence of movement required is not 
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the control handle (or wheel). However, since the input to tracking perform- 
ance also includes kinesthetic feedback from the just-previous movement, it is 
impossible to provide for the learning of wholly adequate. models except by 
giving practice with the use of the specific control used in the actual task. 
This is the reason for the insistence upon “realistic” representation of control 
pressures and movements in training devices and simulators for tracking tasks 
such as gunnery and aircraft maneuvering. It is noteworthy, however, despite 
the correctness of this position, that models acquired for identification of the 
sighting picture alone (without accompanying kinesthetic feedback) may 
nevertheless have the most marked effects on tracking performance. Studies 
of flexible gunnery, for example, have shown the critical importance for good 
tracking of training in recognition of correct sighting pictures (compare 
Goldstein and Ellis, 1956). 

When the identifications involved in tracking can be reduced to simple 
inputs varying in a single dimension, it is obvious that machine performance 
can easily be made to exceed that of a human operator. Extensive training in 
sighting-picture models can thereby be avoided. But the primary reason for 
machine excellence in tracking is one of speed (compare Table 2.1). The 
human individual's time of reaction (not less than 1⁄4 second for each identifi- 
cation) presents a definite limitation to his tracking performance, as a number 
of studies have shown (compare Fitts, 1951). This speed is slow indeed, 
compared to what can be accomplished with the use of such mechanisms as 
servomotors and analogue computers. Since the demand for speed in gunnery 
has greatly increased in recent years, it is not surprising to find that this kind 
of task is to an increasing degree performed by machines, without the partici- 
pation of the human operator. 

The task of flying an airplane constitutes an interesting and instructive 
example of tracking. The tracking required for control and maneuver of the 
plane in flight have long ago been taken over by machine (except, of course, 
for small private aircraft). Even before the automatic pilot was extensively 
used, it became apparent that the identifications required in the course of 
flight were being made with reference to instrument readings, which represent 
extremely simple models for the human being and, in fact, could readily be 
reduced to single-dimension inputs to a machine. The "seat of the pants" soon 
became unnecessary as a source of input for the identifications made in con- 
trolling the aircraft. The situation in landing the aircraft, however, is quite a 
different one. Here the identifications required are based upon models with a 
variety of "built-in" variations, of the sort that are used by human beings in 
perceiving distance. Automatic landing of airplanes can presumably be accom- 
plished, but only when the input variables which establish the requirements 
for the plane's performance can be specified with great precision. Again we 
can realize that this particular tracking task is performed better by a human 
operator because of his ability to acquire and store complex models (of 
distance) which are applicable to a great variety of specific situations. 
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Troubleshooting 


Our account of some typical system tasks has up to now emphasized 
those that involve the function of identifying. It is time to turn our attention 
now to some examples of more distinctly “intellectual” tasks which utilize the 
function of interpreting. One of these is troubleshooting, or the diagnosing of 
malfunctions. While this kind of task occurs most frequently as a part of 
System maintenance, as opposed to system operations, it is important to recog- 
nize that troubleshooting in a more general sense is not confined to main- 
tenance. In many instances, the operation of modern systems requires this 
kind of task as an integral part of “running” the system, or of “putting it into 
Operation.” Strategic missile systems provide an example. While the popular 
image of the “operation” of such a system may conceive of this as beginning 
at the time the missile leaves the launching pad, military users of such missiles 
are well aware that the major portion of system operations takes place on 
the ground minutes, hours, and even weeks before the time of launching. 
The missile and its components are “checked out” in many different ways 
during these periods of time, and even the final countdown before launching 
May be considered to be a standardized checkout procedure. Checking out 
equipment is a task of troubleshooting. y : 

A typical troubleshooting problem is depicted in diagrammatic form in 
Figure 2.5, which shows the situation faced by the radar mechanic in diagnos- 
ing a malfunction whose initial symptom is that no north-south displacemeat 
in Tange is being displayed on the scope, regardless of the value of inputs to 
the north (-south) chain. The difficulty could lie in any of components con- 
Cerned with north-south information, including A, C, G, J, K, and N. The 
Circles indicate check points, and the dotted crosses show where the mechanic 
Would find faulty indications if he were to make these checks. (This infor- 
mation, of course, is not available to the mechanic unless he seeks it.) 

Finding the malfunctioning unit in this case may be done by sequences 
of Checking which vary considerably in their efficiency. One could, for ex- 
ample, simply check at every available point. An efficient procedure, however, 
IS as follows: Point 4 is checked to determine whether both the angle chain 
and the “range chain” are involved. Since 4 is faulty, this meang that the 
trouble must lie at or behind the linked units G and J. This reduces the area 
9f uncertainty to that portion of the equipment which includes ues 4, - S 
and J. Checking in the center of this complex, check point 7 yie ^ a ap 
Teading, which narrows the search to units 4 and C. When a chee z e 
at 6, unit C is found to be receiving the correct input. Consequently, the 


trouble is definitely located in C. 
A task such as this cvidently uti 


functi : Š f 
lons. The mechanic must first o à 
aVailable in his memory the models which represent the components of the 


equipment and their check points, as well as the instrument readings that 


lizes both identifying and interpreting 
all have been trained so that he has 
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P ase chain 


x 


Radar Scope 


Symptom @: No north-south 
displacement in range 


Azimuth 


Figure 2.5. Diagram illustrating an equipment malfunction in a set of 
radar equipment (from Gagné, 1954, by courtesy of Mr. Norman Crowder) 


indicate “good” or “faulty” when he makes the checks concerned. It is note- 
worthy that training in equipment nomenclature and locations and in reading 
test instruments are prominently emphasized features of courses of training for 
radar maintenance. One cannot troubleshoot successfully without being able 
to make these identifications. 

The interpreting function is used at every step of the way in the trouble- 
shooting sequence. The choice of the first check, for example, involves a 
decision among courses of action which are the rules for equipment function- 
ing previously acquired by the mechanic. He responds to check number 4, not 
simply on the basis of its appearance, but in the sense that it is connected by 
rule with the functioning of the combined “angle” and “range” chains. It may 
be noted that these chains themselves do not correspond with the packaging 
of the equipment and are not at all apparent from looking at it. Instead, they 
represent the kinds of conceptual rules which the mechanic must acquire and 
store if he is to perform troubleshooting effectively. Accordingly, training for 
the troubleshooting of specific equipment must concern itself prominently with 
the establishment of such rules in the individual. 

We have no wish here to oversimplify the nature of the dynamic decision- 
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making activity involved in such a task as troubleshooting. Human thinking is 
indeed complex behavior, and the mechanisms that make it possible are by no 
Means well understood. We want only to make it clear that troubleshooting, 
as a typical systems task, requires the use of the human functions of identify- 
ing and interpreting. The individual must exercise filtering and shunting rou- 
tines that take him through the problem in a systematic sequence of steps. 
At one moment he is identifying a check reading (for example, an instrument 
value which indicates “good” or "faulty"), while at the next he is interpreting 
the effects of this reading (for example, "Information is distorted in the range 
chain"). Thus he must have in his memory the models which make identifying 
Possible, the rules which are utilized in interpreting, and the potential pro- 
grams needed for the establishment of a variety of filtering sets. Selection and 
training procedures will need to be instituted taking each of these functions 


into account. 


Flexible Programming with Interaction between Persons 


The operation of total systems usually requires the interaction of people, 
as well as the interaction of an individual man with a machine. As later chap- 
ters will show (see Chapters 11 and 13), the consequences of such interaction 
are to greatly expand the range and variety of inputs to which a given indi- 
vidual must respond, as well as to introduce the necessity for frequent 
reprogramming of behavior. Even at the subsystem level with which we are 
dealing, some notion of the consequences of human interaction may be gained 
by a consideration of load-sharing situations, in which two individuals are 
required to operate in parallel from the same display (or from a duplication of 
the same display). An example may be found in the task of air-traffic control. 

n investigation has been made (Kidd and Hooper, 1959) of the task of 
radar-approach control under high traffic-load conditions, in which the ques- 
tion of load sharing between two operators was explored. The two operators 
received inputs from identical PPI scopes displaying representations of incom- 
Ing aircraft, Their task was to direct aircraft entering the system into one or 
the other of two GCA (ground control approach) gates, using minimum flight 
times and minimum fuel consumption, while maintaining prescribed aircraft 
Separations. The laboratory problem was designed to provide a complete and 
Tealistic representation of an operation involving several aircraft types. Asa 
Part of the experiment, instructional inputs were systematically varied, to the 
Cflect that the operators initially adopted filtering sets of three different sorts: 

rotation control (one controller responsible for every other aircraft); (2) 
Sector contro] (control assigned on the basis of entry); and (3) destination 
Control (control assigned on the basis of landing field). When operators were 
Permitted to transfer control of an aircraft to each other, it was found that no 
“Xchanges were made in the destination-control condition, whereas a high 
Proportion of exchanges were made under both the other conditions, in each 
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case to bring about a destination-control configuration. Thus load sharing was 
eflectively accomplished by the reprogramming of filtering sets by the two 
operators. 

Basically, the task of recognizing and controlling aircraft can readily be 
scen to involve the functions of identification and interpretation. The PPI 
display must first be perceived as a pattern of aircraft having certain positions 
and directions of movement in space, and in relation to runways on the 
ground. The controller must also interpret the situation in accordance with a 
number of previously learned rules, having to do with acceptable conditions 
of heading, altitude, and speed. He is carrying out sequential interpretations, 
and stating vector and descent instructions, so as to insure that the aircraft 
under his control will land in the shortest time possible, using as little fuel as 
possible, and with maximum safety. 

Under reasonable traffic conditions, a single controller would probably 
be able to accomplish this task successfully by adopting a filtering set com- 
parable to "rotation control"—that is, by simply taking responsibility for each 
aircraft in the order in which it appears. But when traffic is heavy, and another 
controller is added to the subsystem, the load must be shared. In such circum- 
stances, it is possible to prescribe the kind of filtering set to be adopted; but, as 
the study we have described suggests, it is not possible to predict which of 
these is going to work best. Alternatively, this aspect of the task may be 
deliberately allowed to remain flexible, and when it is, reprogramming of 
filtering sets will be done by the individuals concerned so as to adopt the sets 
which optimize performance of the task. At a relatively simple level, a kind of 
“system learning" has taken place with these two operators. Their basic skills 
have not been altered; but they have achieved a higher level of subsystem 
performance by discovering what self-instructions will work best in the execu- 
tion of this load-sharing task. 

With a total system, the number and varieties of interactions between 
people are, of course, greatly increased. The possibilities of flexible program- 
ming are similarly increased, not only by the sharing of tasks, but also by the 
performance of tasks in which communication among individuals is required. 
In such instances, flexible programming takes on a new dimension in which 
“self-instructions” become "instructions to other people." There are, accord- 
ingly, greatly augmented possibilities for achieving effective filtering sets within 
each individual operator by means of system training exercises, which will be 
further discussed in a later chapter (Chapter 11). 
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CONSIDERATION OF THE FUNCTIONS OF MEN 
and machines would indeed be incomplete without a 
discussion of that most modern and versatile machine, 
the computer. The development of these system com- 
ponents and their technology has broadened and sharp- 
ened the issues pertaining to assignment of functions to 
man and machine within the system framework. With a 
machine as intellectual as a computer, the decisions about what capabilities 
for specific kinds of tasks can be built or programmed into the equipment, as 
opposed to those which can be assigned to system operators, obviously become 
highly critical decisions, which can markedly affect the success of a system's 
operation. 

The basic functioning of computers is described in this chapter, together 
with the nature of programming, its techniques and tribulations. This is fol- 
lowed by an examination of the problem of allocating tasks to computers and 
men, based upon a comparison of their functions. This in turn leads to a 
consideration of the kinds of system tasks which human beings perform in 
interaction with computers. Finally, the chapter provides a brief account of an 
area of research of considerable current interest to behavioral scientists—the 
problem of simulating intellectual processes by means of computers. 

Perhaps no combination of man and machine could be imagined which 
would pose livelier or more intricate problems of behavioral description and 
analysis than the man-computer combination. The development of computers 
has forced equipment designers to think and talk about machines as though 
they "behaved." As this chapter demonstrates, the behavior of men and com- 


puters can reasonably be considered within a single frame of reference which 
facilitates valid decisions for system design. 
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, Man-machine systems exist to do work. They are worth developing only 
if the work they do cannot be done easily or correctly by men alone. Conse- 
quently, the machines in such systems are usually designed to perform tasks 
which men cannot easily perform. Examples are obvious: bicycles triple speed 
Of locomotion, binoculars may increase by a factor of five or more the effective 
Tange of vision, books increase information storage by unmeasurable amounts. 
Machines which enhance the speed or power of man’s muscles, the accuracy 
Or acuteness of his senses, or the durability and precision of his memory are 


as old as man himself. 

All machines obey or 
extremely limited set of orders, like a 
Constant human control, like a penci 


ders. But most machines either obey only an 
toaster, or gain versatility only by using 
lor an automobile. Perhaps the most 


extraordinary technological achievement of the last century is the development 
Of machines which do better something men do very well: take and give 
Orders, Machines which can sense and execute many diverse orders very 
rapidly and accurately are called computers. We usually think of computers 
aS order takers, but the advent of automated factories illustrates the fact that 
Computers are fully as adept at giving orders to other machines (it might be 
better to say, other parts of the same machine) as they are at taking them 


rom other computers, such as men. 
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Computation, interpreted broadly, is required in any man-machine 
system complicated enough to be worth talking about. And consequently, all 
such systems have computers in them. Two kinds of computers are available 
to system designers: wet or dry. The wet computer, known as the human 
brain, is an extraordinarily rapid, accurate, flexible, and inexpensive device. 
For this reason, system designers until this century have never felt much need 
for any other kind of computer. But in the last 100 years, and especially since 
World War II, engineers and mathematicians have become able to design dry 
computers which for some specific purposes are so miuch superior to the wet 
computer inside the human head that system designers have had to learn how 
to use dry computers effectively. Their efforts have been so successful that it 
is safe to predict that any man-machine system costing more than $1,000,000 
and designed after 1965 will include at least one dry computer. 

How should computers be used? How should functions be allocated 
between computers and men? What kinds of intercommunication devices and 
techniques are most effective? This chapter examines these questions. It begins 
with an expository discussion of computers and computation. It then compares 
men and computers with an cye to function allocation within large systems. 
It next discusses the nature of human jobs in systems which include dry com- 
puters. It concludes with a discussion of computers as models for men and of 
the artificial intelligence problem. 

Much of what follows may sound like science fiction. But computer tech- 
nology is developing so rapidly that the statements contained in the chapter 
are likely to be woefully unimaginative compared with what will actually 
happen. The task of predicting today the future of man-computer systems 
resembles the task of predicting in the days of James Watt, who invented the 
steam engine, the future of mechanical replacements for human muscle power. 


i WHAT ARE COMPUTERS? 


There are two kinds of computers, analogue and digital. An analogue 
computer is one which performs operations such as addition, division, or 
integration on continuously variable quantities, usually voltages or shaft posi- 
tions. Such machines are relatively cheap, can be made fairly accurate, and 
can execute surprisingly complicated orders. Unfortunately, they are very diffi- 
cult, relatively speaking, to give orders to, since the orders must be set up as 
externally wired connections, and they are subject to a number of kinds of 
inaccuracy which limit their usefulness. Nevertheless, they are widely used. 
A computer which is wired into a larger machine for the purpose of executing 
the same computation over and over again for its entire working life is almost 
always an analogue computer. The controls of an automatic washer are a very 
simple kind of analogue computer; an alarm clock is another. Note that such 
machines intermingle digital with analogue elements, a common characteristic 
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of analogue, but not of digital, computers. But when we speak of computers 
we are usually interested in digital computers, and this chapter will be confined 
to them, although some of the things said apply to analogue computers as well. 

Digital computers deal with discrete quantities, digits. The technology of 
manipulating digits by machine is so well developed that almost anything 
which can be done on an analogue computer can also be done on a large 
digital computer, just as fast, much more precisely—and, at present, rather 
more expensively. More important is the fact that logical operations of the 
kind discussed in classes of formal logic can easily be expressed in digital 
form. Most of the important and interesting properties of digital computers 
result from their ability to perform logical operations, or to mingle logical 
Operations with arithmetical ones. In fact, some computer-plus-program com- 
binations are incapable of adding two and two; they are concerned only with 
logical operations. 

Computers are often talked about as though they were chunks of hard- 
ware, like desk calculators. There is an important sense in which the machine 
called IBM 709, for example, is not a computer at all. A computer must be 
told what to do before it will do anything other than take up space and 
Benerate heat or cold. Each computer comes equipped with a language, so that 
Someone who wants to have it execute an order can communicate his desires. 
But that language is so inconvenient for human beings to use that it is often 
used only long enough to provide the computer with some other more con- 
Venient language. A computer plus a language—machine language or some 
Other easier one—can obey orders; the hardware itself cannot. The reason 
Why this point is important is that the computer changes its characteristics 
radically whenever its language is changed. For example, it is often possible 
to program a larger computer like the IBM 704 to behave as though it were a 
Smaller computer like the IBM 650; such a program permits other programs 
Originally written for the smaller machine to control the larger one without 
additional programming effort. The language in which a program is to be 
Written plays a far more central role in programming than does the identity of 
the machine which is to be used, except, of course, for the fact that most 
Programming languages are intended for specific kinds of machines. Hesvalen, 
Whenever the word "computer" is used without further qualification, tetereneg 
toa machine-plus-language system is intended. Ordinarily, all properties other 
than speed, size, and cost are properties of the language rather than of the 
hardware, Of course, the hardware puts restrictions on what languages arc 
Possible or useful, but these are less important than the restrictions on lan- 


Sages which reflect the needs and ingenuity of their inventors. 


Parts or Functions of Any Computer 


The ordinary desk calculator, a simple digital computer. has an input 
Mechanism: an output mechanism; an arithmetic or logical unit; a storage 
; 
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mechanism or memory; and a means for controlling its operation, or control 
unit. The input is a keyboard. The output is a set of digital dials. The arith- 
metic or logical unit and the storage mechanism or memory are the gears and 
cams inside the machine, and in some cases auxiliary digital dials. The control 
mechanism is partly the function keys and partly the operator, who determines 
the sequence in which keys will be pressed. Distinct parts which perform each 
of these five functions are found in most contemporary computers. The input 
devices on most large computers are magnetic tape, punched cards, and but- 
tons on a console. The commonly used outputs are magnetic tapes, high-speed 
printers, and card punches. Most slow-access long-term storage uses magnetic 
tape, although stacks of magnetic disks are also coming into use. Rapid-access 
short-term memory now usually uses magnetic cores which store bits of 
information by means of a hysteresis effect; smaller, slower machines may use 
magnetic drums. Until recently, control and arithmetic units were electronic, 
but contemporary machines use transistors in preference to vacuum tubes, and 
so are sometimes called solid-state devices. 

Computer technology is changing. Computer designers know more about 
how computation can be done than they used to, and new kinds of devices 
with fantastic properties will soon be available. So new ideas about the design 
and organization of computers are likely to make the statements in the 
previous paragraph about the parts of a computer obsolete within a few years. 
Input and output devices will continue to be easily identifiable, but the func- 
tions. of short-term memory, control, and arithmetic or logic will probably not 
be distinct and recognizable. A very highly abstract analysis of any kind of 
computation is embodied in the notion of a Turing machine, which consists of 
a tape, a "read" operation and a "write" operation, which respectively take 
information from and put it onto the tape, and a designation operation which 
specifies where on the tape each read or write operation is to be performed. 


Any set of functions reducible to these three operations could in principle be 
used as the functions of a computer. 


History of Computers 


Of the devices listed above as standard for today's computers, only push 
buttons, punched cards, and vacuum tubes were available in 1940. The history 
of computing is, however, primarily the history of memory devices. The most 
important single development in that history was the invention in 1899 of 
the Hollerith card, the punched card named for its inventor. The punched card 
Stores quite a lot of information permanently without error, gives the infor- 
mation back easily, and can be manipulated remarkably fast, at least by the 
standards of the 1930s. Techniques and hardware designed to exploit these 
useful characteristics of punched cards were developed; devices which sort 
punched cards are still extensively used in business operations. 

In August, 1944, the first automatic calculator was put into operation. 
"Automatic" here means that the sequence of operations which was to be 
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performed was set up in advance, using plug-in wires. Such machines are still 
in use. They are a great advance over desk calculators or sorters for Hollerith 
cards, in that no intervention by an operator is required in the course of the 
machine's work on a problem. But they lack flexibility and convenience and 
can handle only programs of very modest size. 

The idea which makes modern computers possible is the idea of the 
Stored program. It emerged shortly after the end of World War II; John von 
Neumann played a central role in developing it. The gist of the idea is this: 
Since a calculator must have some kind of memory anyhow, it is desirable to 
Store the instructions which the machine is to obey in the same way that data 
are stored within the machine. The instructions are numbers, in fact, and are 
interpreted as instructions because they are called out of memory in the right 
manner and at the right time. A stored program has two advantages over other 
kinds of programs: first, the instructions can be called out and used much 
more rapidly than would be possible otherwise; and secondly, the instructions 
can be modified within the program, during the course of the problem, as 
they are used. ; 

Stored-program machines were a gigantic advance; computer hardware 
development since then has been mostly concerned with making fundamentally 
the same sort of machine bigger, faster, more reliable, and more convenient. 
In 1950, the earliest successful stored-program machines could perform one 
Million additions in about eight minutes. In 1956, it took about 30 seconds to 
Perform a million additions. In 1960 the fastest machines available could 
Perform a million additions per second—and faster machines are on the way. 
Similar improvements in size of memory and cost per operation occurred. 
The upper limit on the number of operations per second in serial computers 
is imposed by the speed of light; information must move around the machine, 
and it cannot move faster than that speed. Computers now in development 


OPerate at about five eighths of that upper limit. 


Advantages of Computers 


i What is gained by using large, fast computers? The most p deg m 
'S speed. It is not just a matter of performing computations in an sah ici 
might otherwise take a week or a month; the improvement is muc more 
dramatic than that. Computers routinely do in an hour or mtn a 
Which would take an unlimited staff of mathematical clerks ol paper m 
Pencil centuries to perform. So computers routinely perform calcu ations whic 
Otherwise would not be performed at all. Furthermore, this M in E 
Speed makes possible the use of what mathematicians call numerica ges s 
to fing answers to problems which do not easily yield to eae " s 
“merical methods are brute-force methods; for example, one e od for 
Solving equations which do not yield to analysis is to try out possi Pier ir 
One by one until a satisfactory one is found. (Most numerical methods are 


T. 
ather more sophisticated than that!) 
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The second gain is in reliability. No machine is perfectly reliable. But a 
present-day digital computer, properly used and maintained, may make errors 
less than once in a billion operations—and if it makes an error, it may be 
able to detect the fact and make appropriate corrections. One of the advan- 
tages men have over computers is a kind of reliability—but that kind of 
reliability has more to do with overcoming the consequences of errors than 
with not committing them. At not committing errors, computers are clearly 
superior to men. 

The third gain obtained from using computers is precision. Within limits, 
the computer can carry as many significant figures as are needed through a 
calculation at no extra cost. The main consequence of this is that many calcu- 
lations can be designed for precise though difficult, rather than imprecise 
though easy, procedures. 

A final gain obtained by using computers is reduced cost. It is often and 
incorrectly argued that, since a computer greatly decreases the cost of com- 
putation, all computations should be performed by a computer. The premise 
is correct only for some computations. Specifically, a computing job must be 
of at least a certain size to be appropriate for a computer; the larger the 
computer, the larger is the smallest appropriate problem. Any computation 
must be programmed, and programmers are well paid. Moreover, almost all 
newly written programs contain bugs (errors); debugging time on the com- 
puter usually costs as much per hour as production run time. A large computer 
like the IBM 709 rents for somewhere between $250,000 and $1,000,000 
per year, depending on what is being rented and who is paying the rent. 
Building, air conditioning, and auxiliary machines cost about $500,000 to 
purchase outright. A staff of 25 to 35 men to program and operate the 
machine and take care of the building is not uncommon; these men plus the 
others who are needed to Support them might cost as much as another million 
à year. If the machine is rented, men to maintain it are provided by the 

s for computers such as the IBM 709 run in the 


D » Such computations may save amounts 
of money literally thousands of times as great as their cost. 
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PROGRAMMING 
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what to do is programming. More precisely, four steps intervene between 
Problem and solution. The first step, usually called analysis, is that of figuring 
out just what the problem is. It often requires collaboration between the cus- 
tomer, who may not know exactly what he wants, and the programmer. The 
Second step, usually called programming, is that of deciding in some detail 
how the computer should go about solving the problem. The third step, coding, 
is writing down the list of instructions which the computer will obey. The 
fourth Step, execution, is the production runs on the computer. Debugging, or 
Correction of errors, occurs at each step, though the word is often used with 
Special reference to correcting errors in coding. 

A program, sometimes also called a code, is simply a list of instructions 
Written in some language the machine can understand; if that language is 
diflerent from the machine's native language, then the program is put into 
the machine along with a previously prepared program which tells the machine 
how to perform the translation into machine language. Certain basic opera- 
tions appear as instructions in almost all programming languages. The mean- 
ings of add, subtract, multiply, and divide are obvious. Transfer tells the 
Machine to look for its next instruction in a place different from the place 
Where it would ordinarily look. Conditional transfer tells the machine to take 
its next instruction from either location A or location B depending on some 
fact it can investigate, such as whether the number stored at location C is 
POsitive or negative. The meanings of read (from the input mechanism), 
Print, and store (in memory) are obvious, as are start and stop. Some of the 
Most modern machines have neither start nor stop available as programming 
instructions, In addition, programming languages always include some kind of 
location or address code, so that the instruction can specity not only what 
Operation is to be performed but also where the raw material is to be obtained 
and what is to be done with the finished product. Characteristically, such 
addresses are simply numbers, which are thought of as serial locations, some- 
times called buckets, in the machine, in which something of value may or may 
Not be stored. A program is a series of numbers stored in a sequence of 
buckets; the machine ordinarily reads and executes the instructions in serial 
Order, unless told otherwise by a transfer instruction. f 

Instructions vary in the number of different addresses which must be 
Supplied in one way or another so that they can be executed. An unconditional 
transfer instruction requires only the address of the next instruction to be 
Obeyed, An instruction to change sign requires only the address of the number 
the sign of which is to be changed and the next instruction address. A condi- 
tional transfer requires two next instruction addresses, plus the address of a 
quantity to be tested in order to determine which next instruction address will 
be used, An add instruction requires four addresses; the addresses of the two 
quantities to be added together, the address of the place where the sum should 


* Stored, and a next instruction address. 
Seldom if ever is explicit listing of all these addresses necessary, or even 
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permissible. The next instruction address is usually simply the next instruction 
in the list of instructions beyond the one now being executed; most computers 
require that it be named explicitly only in the case of transfer instructions. 
There is usually one special location, often called the accumulator, in which 
all arithmetical operations take place. Thus in a single-address machine, if an 
addition is to be performed, one of the two numbers to be added together is 
already in the accumulator; if not, it must be brought there by an extra 
instruction. The address of the other number to be added to the one in the 
accumulator must be provided. The sum is left in the accumulator; if it is not 
to be used in the next instruction, an additional instruction must be used to 
store it elsewhere. Most modern machines are single-address machines. 


Loops 


The most important intellectual tools of computer programming are 
loops, subroutines, and address modification. A loop is simply a set of 
instructions which uses transfer or conditional transfer in such a way as to 
execute the same sequence of operations a number of times, usually modifying 
the addresses each time. For example, consider the following problem. There 
are 100 numbers, Pj, in cells 100 through 199, and 100 numbers, Q,, in cells 
200 through 299. The program is to form the 100 products P,Q; and store 
them in cells 300 through 399. The following loop will do it: 


1. Store the number 100 in bucket S1, 200 in S2, and 300 in S3. 
2. Take the number whose address is in bucket S1, 
whose address is in bucket S2. 
address is in bucket S3. 
Add one to the numbers in SI, S2, and S3. 
Conditionally transfer to instruction 2 
400 and to instruction 5 if the number in 
Go on to the next part of the program. 


multiply it by the number 
» and store the product in the bucket whose 
8. 
4. if the number in S3 is less than 


S3 is equal to or greater than 400. 
3. 


Instruction 1 illustrates the most important feature which all programming 
languages other than machine languages have: 
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garbage and so to use initializing instructions like instruction 1 before using 
them for any purpose. 

Almost all brute-force, or numerical, methods for solving problems which 
are intractable to analysis can be placed in one or the other of two categories. 
Iterative methods use series or other similar approximations. Monte Carlo 
methods, appropriate for use with functions or problems which involve proba- 
bilities, solve the same equation over and over again, using random values 
generated from random numbers, according to probabilities specified in the 
equation. The mean and variance of a large number of such solutions might 
be an appropriate output for some Monte Carlo computations. Loops are well 
adapted to both of these methods, of course; that is why these methods are so 


frequently used in practical computation. 


Subroutines 


The undiscussed major intellectual tool is the subroutine. Suppose a sta- 
tistical computation requires extracting square roots. The programmer will 
have to select some technique for extracting square roots, break it down into 
Component operations which appear in the programming language, and then 
code the program. But he need do it only once. Thereafter, whenever he wants 
to extract a square root within that problem, he need only transfer control to 
the first instruction of the part of the program which he has already used for 
extracting square roots, let that sequence of instructions operate, and then, 
When the square root has been extracted, return control to the next instruction 
Of the unexecuted portion of the program. Such a portion of a program is 
called a subroutine. Subroutines can be generated within a program and used 
again and again, as in this example. But they can also be generated independ- 
ently of any particular program, and either included as a single instruction in 
à programming language or made available as a previously prepared set of 
instructions which can be copied into a program or added as an extra card or 
Set of cards in a program deck. Subroutines can, of course, be used in hier- 
archies; a process as complicated as a factor analysis can be used as a single 
Subroutine, containing many subsubroutines. Some Programming languages 
even permit a subroutine to contain itself as a subsubroutine; a brain-twisting 


arrangement. 


Simplification of Programming 


Progress in computer programming has consisted in large part of develop- 
Ment of programming languages different from, and hopefully easier Y 
than, machine language. Symbolic addressing, loops, and subroutines have 
already been discussed. Their systematic use leads to the development of such 
languages, Many instructions in certain programming languages represent long 
and complicated subroutines in machine language. Almost all programming 
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languages other than machine language use symbolic addressing; the machine, 
rather than the programmer, keeps track of where things are stored. Most 
programming languages use mnemonic operation codes. Instructions in ma- 
chine language may include numbers or letters or both. But in a programming 
language the part of each instruction which specifies the operation to be 
performed is usually part of a word; examples are REA (read), MUL (mul- 
tiply), and TRA (transfer). Many programming languages permit the use of 
floating point notation. This is the familiar scientific notation in which twelve 
million is written 1.2 X 107, or, in the form more appropriate for computers, 
0.12 x 10*. The advantage of floating point for computation is that it makes 
it easy for the computer, rather than the programmer, to keep track of the 
decimal point during calculations and to round off properly when necessary. 
And although most computers use binary numbers, most programming lan- 
guages use decimal numbers; the machine does the translating. 

Mnemonic operation codes, symbolic addressing, instructions not present 
in machine language, and provisions for conveniences such as floating point 
notation are the characteristics of most programming languages. Each such 
language is defined by a “package” of instructions which permit the computer 
to translate from the programming language to machine language; that pack- 
age is usually simply a deck of punched cards or a reel of tape. The translation 
may work in either of two ways. Some language packages simply instruct the 
machine to translate the program into machine language. The machine reads 
the language package and the program and produces an output which is 
simply the program rewritten in machine language. Thereafter, in a second 
operation, the rewritten program plus the data enter the machine, which then 
performs the computation desired. Such languages are called assemblers if 
each instruction in the language in general corresponds to a single instruction 
in machine language. If, instead, each instruction in the language is likely to 
correspond to some large set of instructions in machine language, the language 
is called a compiler. The second way in which a programming language may 
work is that each instruction in the language is interpreted by the language 
package back into machine language and then immediately executed; such 
languages are called interpreters. Again, interpreters may ordinarily generate 
one or many machine-language instructions for each interpretive language in- 
struction, though one-to-many interpreters (such as IPL-V ) are relatively rare. 

Compilers and one-to-many interpreters have in common the characteristic 
that any one instruction in the language replaces many (perhaps as many as 
100) instructions in machine language; the use of such languages is often 
called automatic programming. 'The name is a bit misleading; writing programs 
in these languages is far from automatic. 

To illustrate the amount of effort a Programming language can save, 
Table 3.1 contains five different subroutines all designed for the same purpose 
and for the same computer, but each written in a different programming 
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language. The advantage of powerful compilers like FORTRAN or MAD 
is obvious. 


Table 3.1 
One Subroutine in Several Programming Languages 


The following subroutines look up a particular number in a table 4 of N numbers 
If the particular number in fact appears in Table A, the subroutine reports its index 
(for example, serial position) in the table; if not, the subroutine reports zero. A sub- 
routine for this purpose is written in each of five different programming languages. 


IBM 704 Octal Numeric with no indexing (machine language) 
77 +050000 0 10000 


107 +040200 0 10004 
110 +010000 0 00112 
111 +002000 0 00077 
10003 +040200 0 20000 
10004 +000000 0 00001 


100 +060100 0 10002 
101 +040200 0 10003 
102 4-060200 0 00104 
103 4050000 0 10001 
104 +040200 0 00000 address part to be filled in 
105 +010000 0 00112 
106 +050000 0 10002 
0 
0 


IBM 704 Symbolic Assembly Language with no indexing (SAP) 


CAL N 
LOOP STO COUNT 
ADD CONST 
SLW HERE 
CLA TEST 
HERE SUB ** 
TZE OUT 
CLA COUNT 
SUB ONE 
TZE OUT 
TRA LOOP 


OUT 
ONE DEC 1 
CONST SUB A 


IBM 704 Symbolic Assembly Language with indexing (SAP) 


LXA N,l 
LOOP CLA TEST 

SUB Al 

TZE OUT 

TIX LOOP, 1,1 

LXA ZERO. 1 


OUT 
ZERO DEC 0 
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Table 3.1 (continued) 
One Subroutine in Several Programming Languages 


IBM 704 Formula Translation Compiler (FORTRAN) 


DO 6,1=1,N 

IF (TEST — (A(1)) 6, 7, 6 
6 CONTINUE 

I-0 
7 


Michigan Algorithmic Decoder (a compiler) for IBM 704 (MAD) 


X THROUGH X, FOR I = N, -1, I. E. O. OR. 
A(I). E. TEST 


Every programming language has a name; examples are SOAP, SAP, 
PACT, FORTRAN, FLOWMATIC, MAD, IPL-V. Until relatively recently, 
each language was completely specific to a particular kind of computer. Now, 
however, IBM has FORTRAN packages for its 704, 709, and 7090, and 
packages for languages like IPL-V, which was developed by computer experts 
at RAND Corporation and Carnegie Institute of Technology rather than by 
computer producers, are or shortly will be available for different computers 
not even in the same family. It seems likely that the number of languages will 
decrease as the practice of designing both languages and computers for maxi- 
mum compatibility becomes more widespread. 

Automatic programming is a great time and error saver for the program- 
mer, but it costs heavily in computer operation time. It may take a program 
five or ten times as long to run if it is written in an automatic programming 
language as it would take if it has been written in machine language. For this 
reason, many professional programmers proclaim proudly that they write in 
machine language, rather than any automatic programming language. Of 
course the speed advantage of writing in machine language may sometimes be 
an illusion. Programmer time is at present cheap compared with machine time, 
but debugging on the machine often costs just as much per minute as produc- 
tion runs would, so a machine-language program with many bugs may be more 
expensive than a much shorter, easier program which produces the same result 
but has been written in a highly automatic language. Some programs now in 
use—for example, some of the chess-playing programs—run to 10,000 instruc- 
tions or so in a very highly automatic language. Each instruction in a highly 
automatic language may correspond to twenty or more instructions in machine 
language. Some of these extra instructions are housekeeping instructions 
required by the language, and so would be unnecessary if the program were 
written in machine language, but even so the program might become ten times 
as long if written in machine language as it is in the automatic language. It 
seems unreasonable to set out to write a 100,000-instruction program simply 
to save machine time. On the other hand, some large programs must be written 
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in machine language to reduce their demands on the machine's storage 
capacity, or to save computation time if the program is to be used over and 
over again. Many of the programs used in military systems have this character. 

One relatively recent development in automatic programming is the 
invention of list languages like IPL-V. Such languages use a storage structure 
Which resembles a stack of plates in a cafeteria; only the top plate (bucket) 
is immediately accessible, and as soon as it is removed, the next plate pops 
up into its place. Such a stacked memory structure, of course, is ideal for 
Storing serial lists. The essence of a list language is that its memory can con- 
tain arbitrarily many lists, a list being a variable sequence of items, each of 
Which may be itself the name of another list (or of the same list). Each list 
can easily be changed by free application of operations such as insert and 
delete. It is particularly easy to define recursive processes in terms of list 
Structures. The Information Processing Languages developed by Newell, Shaw, 
and Simon, of which IPL-V is one, are not really designed for mathematical 
applications; instead, their primary purpose is to perform logical manipulations. 


Writing the Program 


The working tools of a programmer are a language and a flow chart. 
Languages have already been discussed. The flow chart is, as its name sug- 
gests, a diagram of the logical design of a program. It looks very much like a 
block diagram of an electronic device. Once a good block diagram of the 
logic of a program has been prepared, the actual coding of the program is a 
fairly routine matter, provided that the problem is neither unusually large nor 
unusually complicated and that the block diagram is not too abstract. This 
description of programming is somewhat idealized, perhaps. Few, if any, pro- 
£rammers customarily prepare flow charts before coding; more often the flow 
Chart is reconstructed from the code, if it is ever prepared at all. 

Debugging is at first a hand operation; for example, flow charts are 
debugged before coding commences simply by tracing information through 
them. Later, hand simulation, which means working through the instructions of 
à program by hand for some simple sample problem, is used to debug the actual 


Program. Eventually, the program must be put kn p epi. if schist 
bugs, t v up at this point. Bugs which stop the prog m- 
C Baa agii d much easier to find than bugs which 


Pletely are much more common an : 

$ ; t 
Permit the program to continue running but cause it to produce ide 
Tesults. And even those bugs which lead to incorrect results rather than halts 


Usually produce such ridiculous-looking errors that they can Sg ipei 
Once; the bug which produces a subtle error is fortunately rare. This 1s : = : 
Instance of complementarity between men and computers; the compu e 
accurate in detail but has little capability for detecting ridiculousness, while 
he Man can often tell whether an answer is “in the right ball park” but 


Usually cannot tell whether or not it is right in detail. 
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One fundamental fact about computer programming outweighs all others: 
the computer does what it is told. Unfortunately, present-day computers never 
do what any fool could tell the programmer wanted; they do what they are 
told to do instead, often with disastrous results. This idiot savant characteristic 
of even the most highly developed automatic languages is what makes them 
hard to use and computer programming hard to perform. The problem is that 
computers do not learn anything from experience, except in the rather 
peculiar sense that they develop libraries of available languages and sub- 
routines. Human beings do not approach each new problem afresh; their most 
heavily used intellectual tool is their ability to apply wisdom gained from past 
experience to the present problem. It is the application of this wisdom which 
permits a man to do what another person obviously meant instead of what he 
said. This restrictive characteristic of computers is evidence that they are still 
in infancy; fortunately, it is not a permanent flaw. Programs capable of learn- 
ing from experience exist, and the computers and languages of the future will 
certainly embody this ability in their interactions with human beings. 


(©) 


THE FUTURE OF COMPUTERS AND COMPUTATION 


Micro Computers 
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basic units of a computer. So powerful is this technique that an expert specu- 
lates that in ten years or less it will be possible to obtain cubes one inch on a 
Side, cach of which will contain 100,000,000,000 (or 10':) distinct units, each 
roughly comparable to a single magnetic core and capable of being separately 
Used as a core is separately used (Shoulders, 1960). The speed of light will 
Still provide a kind of limit on the rate at which such a unit can be used; but 
its small size severely reduces the significance of that restriction, and highly 
parallel computer organization (Holland, 1959, 1960) will reduce it still 
further. A much more serious problem is that of getting information into and 
Out of the elements in that one-inch cube. It is already true that the processes 
of putting information into and out of any large computer are so hopelessly 
Slow that they are seldom done directly. Instead, the punched cards are passed 
through a machine which transfers their information to magnetic tape, the 
tape is then read into the machine, the output of the machine goes into tape, 
and then a separate off-line machine translates that tape into a print-out. Even 
now this procedure is clumsy and only just acceptable; it will be entirely too 
Slow for full exploitation of a micro computer based on units like the one-inch 
Cube discussed above. This area looks like the next serious bottleneck in 
Computer technology. ; T 
Fortunately, a computer with 10'' or more memory units, other internal 
equipment in proportion, and an operation rate near the speed of light need 
Not be used at all efficiently. This means that the computer can be used to 
generate as much as possible of its own input, and to reduce the output toa 
quantity small enough to be printed. Since the information-handling capabili- 
ties of men are more or less fixed, it will be up to the computer—or, more 
Precisely, to those who plan its use—to adjust its input and output require- 
ments downward until it can be in communication with those who wish to 
Use it, Obviously, this will leave room for very highly interpretive languages, 
approaching English in their convenience and flexibility (see Newell, 1960). 
Furthermore, these micro computers will be able to devote large parts of their 
Storage to becoming generally sophisticated about computation, so that they 
Will be able to report back with such statements as “That is ‘ot the most 
efficient way to attack your problem" and to respond appropriately to such 


Instructions as “Oh, hell, you know what I mean.” 


Monsters and Aides 


How will this fantastic capacity and speed be used? It is already clear ie 
Computer design is moving in two quite different directions: large centra 
Computers and small technical aides. A current specimen. of each type is 
Shown in Figure 3.1. The distinction between these two directions may become 
More important as computers get bigger and faster. The growth of nmt 
m. Size and complexity will not be retarded particularly by the development 9 
Micro Machines. If the central data-processing unit is a one-inch cube, it will 
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Figure 3.1. 


A large and a small digital computer. The large computer 
is an IBM 709; the small one is an LGP-30. (By permission of the Inter- 


national Business Machines Corporation; and the Royal McBee Corporation.) 


nevertheless be true that the auxiliary equipment, particularly for input, 
output, and long-term storage, will be very extensive and expensive. As has 
already been mentioned, the larger the machine the larger is the minimal 
problem appropriate for it. Furthermore, the larger the machine, the more 
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work it must have in order to keep busy, and the less time it takes on any 
single problem. Consequently, really large computers will never be common; 
there will be a few of them, in central locations, with lines tying them to a 
ring of peripheral points for problem origination and solution delivery. The 
computer will have elaborate monitoring and interrupt procedures, so that all 
customers get equal treatment and the machine never waits, doing nothing, for 
Some one to make up his mind. Such arrangements, of course, are most 
appropriate for very large continuing data-processing operations; inventory 
control is one obvious example, the census another. 

But large central computers have one serious flaw: it is difficult, though 
not impossible, to use them in real time. ("In real time" is computer slang 
Which means that the computer is made a part of a larger machine, performing 
some tasks whenever they are called for in the course of the larger machine's 
Operation.) Less important but perhaps more annoying, the really large 
machines cost so much per hour that one cannot afford to interact with them 
ina groping, tentative way, and so cannot have a dialogue with the computer 
Or ask it to help in formulating a problem. This sort of need is better met by a 
much smaller machine, which sits in the office or laboratory of the man who 
uses it, available whenever needed. At present such smaller machines have 
smaller, slower memories and less auxiliary equipment than larger machines. 
Whether this will or will not be the case in future depends on the cost of micro 
computing units; it is conceivable that smaller machines may be developed 
Which differ from large ones only in having less auxiliary equipment. 

An exciting concept which combines the best of both kinds of computers 
is the concept of the large central computer connected to many satellite com- 
puters, The satellite computers can perform the functions which small com- 
puters now perform but can also use the services of the large computer when 
necessary. The realization of such a system requires the development of house- 
or the large computer orders of magnitude 
w in use and opens new vistas of difficulty 
languages. Invention of a programming 
for human use, well within the limited 
and nevertheless efficient or capable of 
being used is a stag- 


keeping or executive programs f 
more sophisticated than anything no 
in programming and programming 
language which is easy and natural 
Capabilities of a satellite computer, 
being translated efficiently when the large computer 1s 
Bering task 

els in which the capabilities of future computers may be explored 
is by using them as technical assistants. Newell (1960, pp. 267-268) says: 


It is difficult to differentiate [a machine that is an intelligent adu a 
information] from an extremely compliant, fairly bright human v nica 
assistant, backed up by an impressive computing establishment. X € 
Will converse with such a machine about his problem with the free öm o 
Ordinary technical discourse. . . . The machine will return answers with à 
rather breath-taking rapidity. . . . All our lives are spent learning to liya 
with other intelligences, and relating to an intelligent machine will seem 


more familiar than strange. Since we want machines to help us solve 
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i i are able to make it, the more unobtru- 
agp tee ear ee this help. . . . I do not consider the vision 
radical. Indeed, the programming and computing world is already on its 
way to achieving it.! 

It is reasonable to assume that languages and input-output equipment appro- 
priate to this sort of application will be developed. It has been suggested, for 
example, that the human could write his question in longhand on the surface 
of a desk, and the computer could write its answer below the question. Such 
an arrangement substitutes the very difficult handwriting-recognition problem 
for the perhaps still more difficult speech-recognition problem. 

Whatever the specific configurations adopted, the following predictions 
concerning the world ten years from now seem as safe as any: i 

1. Computers will be far, far cheaper for a given capacity, but their 
manufacturers will increase capacity rather than cutting prices. 

2. Computers will exploit micro techniques by using parallel rather than 
series processes; this means that only capacity and nature of the problem put 
upper limits on the speed with which any particular problem can be solved. 

3. The trend, already highly visible, toward using computers for sym- 
bolic and logical rather than numerical operations will increase, and the 
development of programming languages will reflect the fact. 

4. Very flexible, very easy-to-use programming languages will develop. 
Such languages will have a high degree of ability to resolve ambiguities and 
to exploit vague and uncertain information and instructions. 

5. The meaning of the word “think” will become more and more 


restricted, because our society will continue to define it to mean that which 


computers cannot do but men can, and the list of such capabilities will 
steadily shrink. 


s] MEN VERSUS COMPUTERS 


The system designer faces a choice between men and computers at many 
places in his system. To help him make that choice, many different kinds of 
information are relevant. This chapter will ignore information c 
economic and social problem of technological unemployment and the relative 
production costs of men and machines, all of which are of major importance 
in any real decision of this sort. Instead, it will simply consi 


der the strengths 
and weaknesses of men and computers in the face of tasks which either might 
be asked to perform. 


oncerning the 


Tasks in which Computers Excel Men 


It has already been pointed out that the 


fundamental virtues of com- 
puters are speed 


and accuracy. The performance of many individually easy 


1 Quoted by permission of the National Joint Computer Committee. 
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information-handling tasks, time after time and without error, is not at all 
easy for men; it is what computers do best. Consequently, any high-volume 
information-processing task in which the rules for processing are simple and 
easy is just right for machine performance. And tasks which are not of this 
nature can often be made so by appropriate redefinition. 

One very important kind of task which is not naturally of this high- 
Volume casy-rule nature but which can easily be made so is the task of 
Search. Suppose a man has forgotten the combination to his safe. He will 
Scurry to his files to see if it is written down, and then rack his memory to see 
if he can recall or reconstruct it. But not so a computer. The computer will 
Simply try all possible combinations (which might take a couple of seconds 
to compute) until the right one is found, and then stop. This sort of try-them- 
One-by-one-until-one-fits strategy is the computer's solution to many intellec- 
tual problems. For example, in order to generate a move, a chess-playing pro- 
Bram goes through a series of searches. First, it generates a permissible move. 
Then it examines many plausible sequences of moves which might follow the 
move under consideration, for a sequence length determined by the capacity 
of the machine. It evaluates the position reached at the end of the sequence, 
according to some criterion. It thus finds the best, or perhaps only a good, 
continuation of the original move being evaluated. Having done all this for one 
mOve, it goes on to other possibilities, and eventually selects the best move, 
9r perhaps only a good one. If only computers had enough capacity, they 
Could carry the consequences of each possible move on to checkmate and thus 
Could produce the perfect chess strategy; the amount of capacity required for 
this is far beyond the ambitions of even the micro computer designers. 

The search process, important though it is for any logical operations the 
Computer might wish to perform, contains a serious flaw: most search prob- 
lems lead to unreasonably large sets of things to search. Even a safe can be 
made computer-proof by adding a few additional numbers to the combination, 
The key to getting around this is what has come to be called Dese eet 
Sramming. A heuristic is a rule of thumb which greatly reduces the set o 
things to be searched, and so reduces search time. The cost of henning 
Searches is that if a stopping place is specified, the search may v be pie 
tive, and so may be unsuccessful. A way around this is to make the E 
exhaustive but use the heuristic principles to determine the order in which the 
Possibilities are to be searched. (For more on heuristic programming, see 
Minsky, 1956; Newell, Shaw, and Simon, 1957, 1958b. ) i 

Another task in which computers excel men is long-term Storage; T ss 
remember immense amounts of information, and can reproduce it extremely 
accurately, It is seldom desirable for a man to commit something to dnd 

he can conveniently use a machine perhaps as simple a one as paper 2. 
Pencil. to remember for him. However, machine storage has a serious dis- 


"dvantage, which will be later described. 
8e, which will be later : f 
The other ways in which computers excel men as technical assistants 
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mostly reflect human imperfections from which computers arc frec. Among 
them are: emotional attack on problems, slow speed of learning and necessity 
for repetition, egotism and unwillingness to be treated as a slave, laziness, and 
dishonesty. Note that illness is not listed; computers are often exasperatingly 
“sick”—which means that they do not give the right answer, for no easily 
discernible reason. 

Finally, man-machine systems exist for human purposes, not the other 
way around. No matter how skillfully a machine can achieve an end, that end 
is a human end, not a machine end. This characteristic of computers is in 
principle unnecessary, but in practice it seems unlikely that men will make any 
serious attempt to liberate this powerful race of willing slaves. 


Tasks in which Men Excel Computers 


Many computer experts agree that the most important respect in which 
men excel computers is in the accessibility of the items in storage. Men can get 
at a single memory in many different ways; in particular, they can recover 
memories on the basis of similarity alone. Computers, by contrast, have no 
such efficient cross-indexing. If they did, it would be possible to write pro- 
grams which rely on the computer to locate and produce any item in memory 
without specific instruction concerning where that item is. At present, no such 
procedure is possible. Cryogenic computers will have capabilities of this sort 
which excel those of current computers, but they will still be much inferior 
to the capabilities of men. 

Psychologists and computer experts would also list pattern recognition, 
particularly visual pattern recognition, as an important capability in which 
men far excel computers. A great deal of research on how to make machines 
recognize patterns is in progress. The fundamental difficulties result from the 
fact that men recognize patterns in spite of transpositions, rotations, transla- 
tions, and many other varieties of systematic and random distortion. Machines 
cannot do nearly so well: it is a great achievement that Selfridge (1955, 1956, 
1959) has been able to train machines to recognize Morse code sent by 
human operators. Note the use of the word "train." The assumption under- 
lying almost all attacks on the pattern recognition problem is that the complex 
structure of equivalence classes of stimuli necessary for pattern recognition 
must be learned, rather than built into the machine. There are even some 
comparatively unsuccessful attempts to train machines by means of reward 
and punishment, in the absence of any specified structuring of the machine 
(Friedberg, 1958). At any rate, the problem of pattern recognition is very far 
indeed from being solved, in spite of the strident newspaper stories concerning 


various pattern recognition programs which appear from time to time. Until it 
is solved, computers will be unable to inter 


€ pret complex visual displays such 
as aerial phot: 


s ographs, will be unable to Tecognize spoken words, will have 
difficulty recognizing longhand, and in general will be markedly inferior to 
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men in a crucially important technological skill. But this deficiency may well 
not exist 15 years from now, in view of the rapid rate of progress now being 
made in research on mechanical pattern recognition. 

A major virtue of men is that they have a high tolerance for ambiguity, 
Vagueness, and uncertainty. Men are able to detect what other men mean 
through the smog of what they say, and they customarily do so and behave 
accordingly. Such tolerance for ambiguity is based on a life-long history of 
experience with ambiguity and on the ability to argue by analogy from one's 
Own purposes to those of other people. Neither of these characteristics seem 
likely to be available for computers in any near future. So long as computers 
cannot tolerate and exploit ambiguity, they cannot be given major executive 
responsibilities unsupervised; social control is usually based on vague man- 
dates which permit wide but not unlimited latitude in interpretation (for 
example, platforms of political parties). This means that man-machine systems 
Will necessarily continue to have men with veto power over computer- 
Senerated decisions, rather than vice versa. T ae da 

One reason why men are good at tolerating and exploiting ambiguity is 
that they can effectively translate uncertainty into probability—another task in 
Which men far excel computers. Consider the statement, “Before you go to bed 
tonight, you will consume a bottle of beer." Presumably that statement is 
neither impossible nor certain. A computer could probably go no farther; a 
man can attach a number to the statement which represents his evaluation of 
its Probability of being correct. Such numbers are, it turns out, excellent guides 
to action; men can accurately translate uncertainty into probability. Com- 
Puters, on the other hand, are far superior to men in taking probabilities and 
Payoffs and computing from them the best course of arnon: These considera- 
tions suggest that a military-information processing system which must cops 
with relatively unreliable data (such as a sonar system) i id il use 
human operators as transducers for probabilities. These pens ities E 
be entered into a computer, which would then compute the apne anes = 
action in the light of them. No such T now exists, but it seems entirely 

ossi at there might be one ten years from now. 
P E el cnr p o competition ePi cd pie 
important, are rather remote from current system applications. The E 
to be mentioned are crucial for most current man-versus-computer problems 
In man- ine systems. , , 
dej cibis ani is that in a very important sense msi are tar duet 
Teliable than computers. It has already been pointed a that P r i 
far fewer mistakes than men. But in general the mistakes compu id de 
Sither remain unchecked or stop the computer completely. Man, on we other 
hand, can detect his own mistakes and spontaneously work outa 2: to 
Correct them or remedy their effects. Furthermore, once he kas death : ow 
to Perform a task correctly, man does not repeat. and repeat thie same Bs 
as will a computer with a broken part. In short, if a little allowance is made 
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for the approximate nature of human reliability, man is far more reliable than 
any computer yet invented, or any likely to be invented in the near future. 
Of course, some computers are to a degree self-correcting even now, and most 
large future computers will embody various kinds of self-correction features. 
But the versatility of human beings in dealing with error is an extremely 
remote goal indeed for computer technology. This human reliability char- 
acteristic is so important that major systems are likely to continue to include 
human components for a long time to come for this reason alone. 

Perhaps the most practical advantages of men over computers, of course, 
are in cost and availability. Consider a system which has an expected life of 
ten years. Over that period, it might cost $120,000 in salary to employ one 
man to tend the system. The system for personnel support and other indirect 
personnel costs might over that time cost another $120,000. That puts the 
cost of one human brain, for many purposes the most effective computer now 
available, at something like $240,000 over a ten-year period. By contrast, an 
IBM 709, for many purposes much inferior to the man, costs something like 
$1,000,000 per year, or more than 40 times as much as the man over the 
ten-year period. Even if the system were to operate 168 hours per week, which 
requires at least five men for each operator position, the computer would be 
much more than 10 times as expensive (since the cost of supporting it would 
go up with round-the-clock usage). Of course, the 709 can do many things 
that men cannot do, and vice versa; which is preferable depends on the 
specific nature of the job to be performed. But the man has a powerful built-in 
economic advantage, which is not likely to change very much with time. 

Availability is important also. There are many men, and at least à 
third of them can be trained to perform difficult intellectual tasks. There are 
few computers, and the number of really large ones is unlikely to become as 


large as, say, one millionth of the number of men. So men not only are 
cheaper, they are also easier to obtain. 


[aj 


OPERATOR JOBS IN A MAN-COMPUTER SYSTEM 


How can a system be designed to exploit the special abilities of men 
and of computers? The art of designing man-machine systems which include 
computers is too new to have very many well-understood and generally 
accepted principles. Nevertheless, most military information processing sys- 
tems which include computers have a family resemblance to one another. 
Some unsystematized wisdom has evidently accumulated about how to design 
such systems. 

Communication of that unsystematized wisdom is difficult, because parts 
of the detailed designs of every military information processing system are 
classified. Release for publication even of descriptions of such systems which 
contain no classified information is difficult or impossible to get. In an attempt 
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to get around these difficulties, the following paragraphs contain a somewhat 
detailed description of an imaginary military system. The system setting, naval 
hunter-killer operations, is real, and the ideas presented are, in fact, drawn 
from experience with a number of real military (but not naval) information 
processing systems. All the technical ideas presented have previously been 
included in nonclassified technical publications; only their integration into a 
System context is original here. This means, of course, that no classified 
information is presented. Of course naval hunter-killer groups do exist and 
must be controlled somehow. But the system for controlling them described 
below deliberately violates some Navy policies concerning allocation of 
command functions, and so would be unacceptable to the Navy. 


The HUK-Master System 


The following paragraphs describe the HUK-Master System, an imagi- 
nary Navy system assumed to have been designed in 1952-1954, to have 
Passed tests of a prototype in 1958, and to be becoming operationally avail- 
able in 1960-1961. (HUK is Navy slang for hunter-killer. A hunter-killer 
Operation is one in which a number of ships, planes, and possibly submarines 
Collaborate to detect and destroy enemy submarines patrolling coasts or 
Shipping lanes which are to be protected.) M ; 

Since the system is carried on board a ship, it must be relatively small. 
For this reason it uses a number of relatively small analogue computers rather 
than one large digital computer, and many operations which in larger Systems 
might be automated are in this one performed by men on line. The following 
description is confined to operator positions; of course large numbers of mainte- 
Nance and support personnel are required in connection with any large system: 

The HUK-Master System receives information from long-range sonar 
devices located on land, on its ships, and on sonar buoys—as well as infor- 
mation about courses and speeds of friendly ships and submarines 794 intel- 
ligence information about enemy activities and plans. This sonar * — 
in general reports the location, course, and speed of objects whic! e E 
enemy submarines—or might be friendly ships or submarines, or w : ie 
Schools of fish, or layers of shrimp. It attempts to identify these — i. 
tracks those which are not identified as friendly. It sends planes vA ip x. 
investigate possible enemy submarines, providing information whic! ate ed 
Signed to enable the investigating ship or plane to find the proper a dps 
then to acquire the unknown object on its own sonar or other senso: To 

eps track of attacks by its controlled ships and planes, their ay m 
"C, and initiates further action with respect to enemy submarines i i ae 

The heart of HUK-Master is its central station, located coach = 

Partments on the largest ship (a small aircraft carrier) in =r € r is 
© main room is a large one, designed somewhat like an ampi it ea 
è large manually operated display where the stage would be, a set of auxiliary 
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status boards, and a number of operator positions, each one having a 
computer-controlled cathode ray tube display and a complex console. The 
illumination of this room is blue, because the dim orange dots on the opera- 
tors' cathode ray tube displays are best seen if the over-all illumination is 
dim and blue. The large stage display is made of edge-lighted transparent 
plastic; marks made on it with grease pencil show up in vivid color in the 
edge lighting. 

The first important. operator position within this room is that of the 
entry operator. He first instructs the computer to initiate a track on an uniden- 
tified moving object which appears in a long-range sonar return. Next he tries 
to identify the object. The first question is: is it on the surface or submerged? 
Comparison of the sonar return with a radar return or the results of a visual 


to one of several tracking operators. He may monitor the performance of any 
tracker, and sometimes moves tracks from one operator to another in order 
not to overload one operator. He also obtains information about the pattern 
of temperature layers in the water at the location of the tracked object from 
àn operator who has custody of this information; this information permits 
Some inferences to be made (by the computer) about the depth of the object 
being tracked. Finally, the entry operator keeps track of what happens as a 
result of any attack initiated by the system against the object, and he can, if 
necessary, propose further action to the tactical Officer. All these operations 
s phones which permit one operator 
e intended primary channels of com- 


an look at either live sonar returns or 
computer-generated symbols or both, at his choice. 


taining the track, and uses his much 
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The next operator job, perhaps the most difficult in the system, is that 
of the tactical officer. He must decide which ship or plane should investigate 
Or attack which object (a decision often suggested by the computer in larger 
land-based Systems) and inform the computer of the decision. Of course he 
Must first present his decision as a proposal to the task force commander, 
and that commander must approve it, but such approval will in general be 
routinely given. (Only the fact that ships and submarines move relatively 
Slowly permits this time-consuming on-line monitoring operation. In Systems 
designed to control aircraft or missiles which are attacking other aircraft or 
missiles such an arrangement would not be possible.) Once the decision has 
been made, the tactical officer informs the computer about it. The computer 
informs the ship, if a ship is to investigate or attack, or informs the aircraft 
Control center, if a plane is to investigate or attack. The ship or aircraft con- 
trol center is provided with a display of estimated location of the object, based 
9n its HUK-Master track. The ship’s captain or the aircraft controller can use 
this information to vector the ship or plane to the right place and to enable 
the Ship or plane to pick up the object on its own sonar, or magnetic detection 
Scar, or other sensor. When the attacking unit sees the object on its own 
Sensors, this information is passed on to the HUK-Master computer, which in 
turn informs the tactical officer. If the ship or plane has difficulty in locating 
the unidentified object, its officers can talk directly by radio-telephone with 
the HUK-Master tactical officer. The tactical officer has no access to live 
Sonar displays; he sees only computer-generated symbols. The same is true of 
the co UK-Master. 

Face, thes d of course a commander of the HUK-Master System; he 

as a small command post of his own, which overlooks the main room in the 
Central station. He has no specific responsibilities during Operations other than 
that of general supervision of the system. In this system as in many others, 
the commander has elaborate displays to look at, but his command post is 
Separate from the place where the work is being done and his only means of 
generating an output is a telephone. As e Meg ^i processing Sys- 
tems become more complex, commanders have fewer and fewer inicia to 
intervene in the actual operation of the system. After all, the ‘sei ge 
Such system exists in the first place is because the aaa re m E 
*cisions is too complex and too abundant to be mne uri Harre 
the heads of a few men. For the commander to intervene A ded ca Mur 
Perform with little help the task for A à mpm Epam cpm 
Perform j atisfactorily than the syste ioc E 
the momen Cais if the system is very seriously eri on dips m 
© be able to do so. In effect, in the more complex modern "o 
Much of the command function has been ei rapi comp a “eg 
*Ystems, and in particular to the computers within those a id Me re 
tasks inherent in the command function in these a p T 
Uman capability. Of course this process has gone muc y 
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which must deal with fast-moving objects such as aircraft or missiles than it 
has in slower systems dealing with ships and submarines, or with columns of 
trucks or tanks. It is very unlikely that antimissile missile systems, for ex- 
ample, will make any substantial use of men on line; such systems must act 
so rapidly that human delays will not be tolerable. So the major functions of 
commanders in most future systems will come increasingly to consist of bear- 
ing ultimate responsibility for system performance, managing the people who 
operate the system, and managing the personnel support system behind them. 


Design Principles for Systems which Include Computers 


Computers and men perform little-understood functions in competition 
with each other, and the System designer must, for each function included in 
his system, resolve the competition in favor of one or the other. Unfortunately, 
the necessary scoring system is hard to find; these functions in human beings 
are not well understood, and in computers the score is constantly changing as 
technology advances. Still, some principles appear to be relatively valid and 
stable guides to good design practice. 

The first question the designers of a new System must face is whether or 
not the system should include a computer. The sheer economic considerations 
already discussed are important but not conclusive; social costs, such as 
technological unemployment and the cost of training system operators and 
technicians, must also be considered. Once the decision to put a computer 
into the system has been made, several general principles concerning the 
design of computerized Systems can be used to guide design. Some of these 
principles are illustrated by HUK-Master; some are not. 

1. Allocation of functions among men and computer should consider the 
best, skills of each. It is seldom wise to allocate to the computer everything 
the designer knows how to mechanize, and to parcel out among the system 
operators whatever is left over. 

2. If possible, the computer should be about 80 per cent used; if it has 
too much unused time or capacity, either a smaller computer should be used 
instead or tasks for which it is less than ideally suited (such as long-term 
memory) should be given to it. Computers do not profit from rest periods, 
other than the necessary halts for maintenance and repair; men do. On the 
other hand, tasks change, and a little flexibility is therefore desirable. 

_ 3. Provision of one sort or another must be made for system function 
during computer malfunction. This often implies either a second computer or 


a manual back-up system. In some Cases, of course, no meaningful provision is 
possible or worthwhile. 


4. Operator jobs should no 
should require a relatively high le 
This reduces the requirement for 
Structure within the system. 


t be homogeneous in difficulty. Some jobs 
vel of ability and training; others should not. 
high-IQ operators and provides for a career 
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5. Man-to-man communications should be relatively rare. In general, 
human outputs should go into the computer. The process of man-to-man com- 
munication is so often clumsy and imprecise that the system is usually better 
off if any two functions performed by different men are separated by a 
function performed by the computer. This principle is controversial; some 
experts insist that a great deal of informal man-to-man communication is 
both necessary and desirable. 

6. Men should function as aids to the computer in sensing, extrapolating, 
and decision making. The entry operator in HUK-Master senses which spots 
of light on a sonar return are really ships, decides which of the ships should 
be tracked, and so informs the computer. The tracker helps the computer 
When its mechanism for extrapolating is inadequate for the job. In each of 
these functions, the computer can be given the primary responsibility so long 
às a man is ready to back it up when it fails (though this is not done in HUK- 
Master for the sensing and decision-making functions). The idea of using men 
as back-up systems for computer functions is very widely applied; most 
manned space vehicle designs are designed that way. There is some question 
Whether in many applications it might not be cheaper and just as effective to 
have the man perform the function in the first place. . 

7. If at all possible, the computer rather than aman should have primary 
responsibility for maintaining vigilance and detecting when, after a period of 
inactivity, some system action is required. . . 

8. A number of specific tasks which must be performed in most infor- 
mation processing systems are usually allotted to men because they use man’s 
best skills. The functions of detection, identification, and tracking have already 
been discussed; they exploit human pattern-recognition ability and ability to 
Cope with uncertainty. Another common function is goal-setting for searches. 
Computers very often solve problems by means of iege NN xx e 
Very large set of possible solutions. Searches shoul hee e pa e üy 
hypotheses concerning the most fruitful places to "€ irst—t at m y 
Using heuristic principles; it is arbitrary whether searc và ssi ne e a 
Solution is found or whether it stops after the most likely possi in ; ave 
been searched. A computer program can induce these mene i s —— 
experience with the problems, but more often in caede, emm As a 
will supply computers with heuristic principles n gui penes die pere 
this sort of technique has so far been made in mi gp Noes IN he jim 
Systems, but they will obviously be very aon aton vidi will n 
Of systems for processing the immense volumes of in 


8enerated by reconnaissance satellites. 

9. Yet another important human fu 
censorship. Men monitor the output of com ci Ke iui 
Computer actions when it seems appropriate to do des oe : 
Systems get more complicated and their tasks become de: iai 
Will be more and more difficult for men to censor system outp' 1 


nction in computerized systems is 
puters, with responsibility to veto 
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They cannot assimilate enough information to be sure whether the system is 
right or wrong, except in the case of gross malfunction. More important, 
systems can seldom tolerate the response of doing nothing, and men often 
cannot accumulate the information or the time to supply alternatives to the 
computer’s recommended course of action. So the veto function will become 
more and more symbolic, and the commander’s telephone line will play a less 
and less important role in the system’s functioning. 

10. It will continue to be true that systems which include computers 
exist to serve human purposes, so system goal-setting will continue to be a 
human function, the most important human function in the system. However, 
that function will be performed mostly by the designers of the system and 
those who write the computer program; the nature of system design pretty 
completely determines the goals which it can effectively further. 

The most serious problem which can arise as a result of using men as 
on-line components in a system which also includes a computer is caused by 
the severe mismatch in rate of performance between men and computers. 
Computers ordinarily perform their operations very fast indeed, while human 
beings are ridiculously slow by comparison. To make a computer wait while a 
man performs some function on line is extremely wasteful of computer time. 
It is worth doing only if the function cannot be satisfactorily performed in 
any other way. The assumption which underlies the preceding discussion is 
that cases in which that penalty is worth incurring are fairly frequent. Some 
psychologists and computer experts disagree, arguing instead that men should 
perform only off-line functions such as goal-setting, monitoring, and censor- 
ship. Some systems must function so quickly that human bottlenecks cannot 
be tolerated; it seems probable that antimissile missile systems would fall into 
this category. But for most slower systems, and in particular for most civilian 


systems, the advantages of exploiting human skills far outweigh the disadvan- 
tages of putting up with slow human reactions. 


The Man-Computer Interface 


Any man-machine system must provide displays and controls for its 
operators. In a system which includes a computer, such interfaces between 
man and hardware must be very carefully designed. Unfortunately, there is a 
serious mismatch between what is best for man and what is easiest for the 
computer to use or generate; much design effort must be devoted to mini- 
mizing the consequences of this mismatch. 

Computer-controlled displays are usually presented on the face of a 
cathode ray tube (CRT). Special equipment permits combination of blips or 
spots, outline maps, and alphanumeric information on the same tube face. 
Such displays nicely fit some characteristics of c 
tronically generated and therefore easy for the 
They can change as rapidly as the computer c 


omputers. They are elec- 
computer to control directly. 
hanges the information to be 
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displayed. And they are well adapted to what psychologists call contingent 
programming. "Contingent programming" is a phrase denoting the simple idea 
that the operator can control what information his display presents to him. 
It is well established that operators can use such capability to reduce the 
amount of irrelevant information presented to them. Unfortunately, experience 
indicates that operators often prefer to look at more information than they 
need, and that the excess information actually hinders them. Thus many 
Systems, including HUK-Master, restrict the classes of information at which 
any one operator can look. 

CRTs with many different kinds of phosphors are available. This means 
that many different colors, brightnesses, and persistences are available to the 
designer. The question of persistence is important. A short-persistence phos- 
phor rapidly fades when the beam of electrons which excites it is turned else- 
where; this means that the same information must be put onto the tube many 
times. On the other hand, a long-persistence phosphor, though it preserves 
information longer after each application, cannot change nearly so fast when 
the operator wishes to look at something different. Systems which include an 
auxiliary storage device, often a magnetic drum which receives from the 
computer the information which is to be displayed on the tube, can alleviate 
but not cure these difficulties. . . 

No military system now available uses multicolor CRT displays. Color 
Coding is less important than one might suppose; only somewhere between 
four and eight absolutely identifiable distinct colors are possible, and the 
same sort of coding can often be achieved by contingent programming. Never- 
theless, it seems likely that the development of color TV technology may have 
as a by-product the development of color CRT displays for military systems. 

No tube-generated display can include as much detail as, for example, 
a photograph. Thus photographs, as well as drawings such as maps, are likely 
candidates for inclusion in systems where rapidly changing information which 
does not demand high resolution is blended with relatively static high- 
resolution information. One way of doing this is to project a color slide onto 
the front of a CRT, simultancously producing blips or other similar displays 
from the rear. Such displays, having the potentiality of combining € 
aspects of two systems, are now coming into usc. Elaborate Een or 
handling slides mechanically under computer control are now under develop- 
Ment for just such applications. — . 
. . One interesting possibility aygilable in so ile ge 
is time compression. If information changes 9r? k d 
it may be helpful to the operator to be able to review the history of 3 Aoc 
rapidly, perhaps looking at each display generated during “7 past «imei 
Or a second or two. If cven faster time compression is used, appare a 
Ment like that obtained in movies or TV may occur, this may help to dis- 
tinguish signals from clutter. Recent research suggests that for many pe int 
à multiple-exposure photograph may be a better time compression device 


me computer-controlled displays 
ery five or ten minutes, 
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than a serial presentation of successive times; equipment such as the Polaroid 
Land Camera permits multiple-exposure photography without significant 
processing delay. 

Computers can, of course, control other kinds of displays. The only 
important other one, however, is the print-out. This can effectively be used 
for system purposes. Onc interesting possibility is the direct printing of map- 
like information. 

The fundamental nature of the display mismatch between man and com- 
puter is simply that man cannot deal with displays which change in time nearly 
as fast as computers can generate them. He can use detailed information at a 
given time more effectively than computers can usually display it. This is 
especially true in CRT displays, with their inherently poor resolution. Thus, 
much of the art of designing computer-controlled displays consists of reducing 
the time-variability of the display and, if possible, of trading space for time. 

Continuous control systems, like those which might be used to control a 
high-speed submarine or a hovering helicopter, present problems which can 
be attacked by using computer-controlled displays. A submarine, for example, 
has great inertia in moving up or down; it is often hard to get this movement 
started and often even harder to stop it once the desired depth is reached. 
Such characteristics, which produce complex relationships between what is 
done to the operator's controls and what the System being controlled does in 
response, are called control dynamics. When the control dynamics of a system 
produce severe control problems, computer-controlled displays can often 
alleviate them. One class of displays useful for this sort of purpose is called 
the quickened display (see, for example, Taylor, 1957). Quickened displays 
are concerned with error signals for rapidly changing continuous-control tasks, 
Such as controlling the hovering of a helicopter. They display a combination 
of error and the first derivative, and possibly higher derivatives, of the error 
signal. Thus they permit the operator to change his control setting when the 
rate at which error is accumulating starts to rise, before the total amount of 
error has changed seriously. A quickened display looks very jittery and 
changes very fast; one might suppose that it would be exceedingly hard to 
use, and it is true that operators do not particularly enjoy using it; but when 
used appropriately, it produces error scores orders of magnitude lower than 


display i 
that any action of the control will 


ystem will be doing at a series of future times 
(for example, 1, 2, 3, . . . 60 seconds ahead) if the control is immediately 


returned to a null state. Thus the Operator can rapidly change his control 
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setting to one which produces the desired state of the system a minute or so 
from now if the control is returned to a null state, and then do so. The advan- 
tages of predictor displays, in suitable situations, are even greater than those 
of quickened displays—and operators love them. Figure 3.2 is an example of 
Such a display. 

There is a much smaller degree of mismatch between man and computer 
when information is going from man to computer than when it is going from 
computer to man. The push button is an entirely satisfactory technique for 
communicating human decisions and instructions, provided that only a few 
items of information need be transmitted, and the stick is a very satisfactory 
device for inserting continuously varying information. One practical problem 
of some importance arises when an operator wishes to designate a particular 
Object on a display when giving instructions to a computer. The old-fashioned 
Way to do this was to use cross-hairs or some other movable marker attached 
to the computer. A more modern way of doing it is to use what is called a 
light gun—an object shaped very much like a pistol, which contains a photo- 
electric cell. When placed over a blip on a tube face, the photoelectric cell is 
excited by the pulse which renews the blip. The exact time when that blip was 
renewed is detected by the circuitry controlled by the light gun. Since only 
One blip is renewed at a single instant, the computer can thus figure out Which 
object the operator had in mind. This is a clumsy device, only slightly less 
So than cross-hairs or an electronic cursor. New systems use smaller, less 
bulky, light pencils, but the task of putting the light pencil exactly on the 
blip is still a nuisance; it is to be hoped that a better solution will be found. 

Of course, in controls, as in displays, there is great sum ibe med 
in designing computer programs and auxiliary equipment to a E 
effectiveness by using computer capabilities to make human wb E E 
the only important idea of this sort that is apparent at orsa emt 
making one push-button initiate a sequence of events. Surely g g y 


Ought to be possible in the future. 


The Future of Man-Computer Competition 

even the fantastic progress anticipated in 
tory in the man-computer 
d computers differ suffi- 


It should be clear by now that à 
Computer technology will not permit a clear-cut vici 


Competition, It will continue to be true that men an pe a 
ciently from one another in skills so that large systems WI . 


a T 

Not about to become obsolete. Therefore the future of Fa man a e 

Competition lies in symbiosis rather than in extermination, E n i 

Will be designed to exploit harmoniously the best virtues of bo i E 

nological basis which the system designer = m B 
appy marriage is a clear understanding of the capabilities © 

Computers, iud detailed knowledge of the mechanisms available for use at 


© interface between them. 
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( COMPUTERS AS MODELS OF MEN 


Although it is not really crucial for system design, a chapter about men 
and computers would be incomplete without a discussion of the old, emotion- 
laden, unresolvable question: can machines think? 

In an important sense, the question is meaningless. Contemporary lay 
definition of “thought,” or, for more sophisticated laymen, "creative thought,” 
usually is: what man can do but a machine cannot. By that definition, 
machines cannot think and never will be able to, no matter how able they 
become. Unfortunately, that definition is difficult to replace with a more 
meaningful one; psychologists have never been able to decide what thought 
is or how it can be recognized when a man or a machine does it. The whole 
question of what thought is and whether machines can do it has become so 
€motion-laden and so meaningless that computer experts who are engaged in 
enabling their machines to think have now widely chosen to circle around it 
by calling their field the creation of artificial intelligence. From this point of 
workers are attempting to solve is the problem 
of designing machines which have particular valuable intellectual skills which 
intelligent animals (including men) also have, such as pattern recognition and 
Problem-solving ability. It seems safe to say that any intellectual process 
recognizable in man or animal can now be simulated, though often very 
crudely, by a computer program. It is out of research on simulation of human 
intellectual processes that much of the progress 1n computer technology, par- 
ticularly in programming, is coming; it is reasonable to hope that this research 


Vill be even more fruitful in the future. 


view, the problem that these 


Simulation of Neural Structures 


Three main streams in research on artificial intelligence can be distin- 
latior imulation of intellectual and behav- 


guished: si i 1 I tures, si 
` : simulation of neural struc 1 1 ipe é 
‘Oral processes, and applications for artificial intelligence. Of these, simulation 
Of neural structure may possibly be the least promising, although it is the 


farthest advanced. Such programs usually take Hebb's (1949) sage of 
OW the nervous system works or one very much like it, mun that m 
ception by creating within the computer a network of semai. E - 
TOutines) which behave like neurons, and then attempt to show that = 
Network can exhibit at a very low level some intellectual functions, most often 
Pattern recognition, Rosenblatt’s Perceptron (1958a and b) is 2 pupam 
Which does this with a substantial amount of structure initially applied by the 
Program. Holland (1959) is attempting much the same with an initially quite 
Unstructured nerve net; his main interest is the emergence of functional 
Organization among the “neurons.” No real attempt is made to simulate any 
"ey complicated process; no simulated nerve net has so far contained more 
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than 10,000 neurons. It is unlikely that computer technology will in any 
reasonably near future be able to supply elements remotely as versatile and 
abundant as human neurons, so it is unlikely that we will even be able to 
evaluate this approach soon. Of course, the neurophysiologists find it very 
valuable thus to be able to simulate their more general ideas about how the 
nervous system works; this technique is therefore a very valuable rescarch tool 
in neurophysiology, even if not in the study of artificial intelligence. (See 
McCulloch et al., 1956; Milner, 1957; and Rochester et al., 1956.) 


Simulation of Intellectual and Behavioral Processes 


Research which attempts to simulate intellectual and behavioral processes 
seems to be divided into two kinds, each based on a particular viewpoint about 
Psychology. One kind starts from the radical behaviorist’s view that the only 
kind of process which matters in behavior is the fixation of responses as a 
result of reinforcement. In artificial intelligence, this has taken the form of 
evolving programs by random processes. In effect, the computer is provided 
with an initial random set of instructions, combined with a feedback mecha- 
nism which evaluates the product of this random set and administers a 
reinforcement making successful instructions more likely to occur and unsuc- 
cessful ones less likely to occur (compare Friedberg, 1958). It has been 
demonstrated that such a machine can, in fact, develop a program which will 
achieve a desired effect, starting from a random initial set of instructions. 
Unfortunately, it took about 300,000 reinforcements to be successful. That 
remarkably large number can be reduced somewhat (perhaps to 100,000) by 


more efficient management of the basic idea, but there is no reason at the 
moment to suppose that it will 
to be an effective means for de 


With a loudness proportional to the 
also proportional to the outcomes 


If that decision was Correct, that demon 


t weights still more highly the tests 
which came out favorably and still less 


highly the tests which came out 
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unfavorably on that Occasion; if the decision was incorrect, it weights less 
highly the tests which came out favorably and more highly those which came 
Out unfavorably. Thus in due course a demon whose tests are optimally 
Weighted for the particular diagnostic category which he represents is evolved. 
^ more elaborate version of this general idea would presumably use higher- 
order demons to weight differentially the outputs of the first-order demons, 
and so on. The program can also be designed to invent and evaluate new tests. 

Newell, Shaw, and Simon’s (1958b) General Problem Solver is simply 
a search program which uses some very general heuristic principles and which 
is Capable of modifying its heuristic principles or developing new ones. It 
evolved out of an attempt to develop a program which would invent proofs 
for the theorems about logic found in Chapter 2 of Whitehead and Russell’s 
Principia Mathematica (1925). It recognizes three fundamental entities: 
expressions, differences between expressions, and operators. The program 
attempts to apply permissible operators to expressions in such a way as to 
reduce to zero the difference between two expressions which might be called 
the original problem and the solution. Thus the program can be used to solve 
Problems of the form “Given expression a and a set of admissible operators, 
derive expression b.” It does this by means of goals, subgoals, and methods. 


There are three types of goals: 


Transform Goal: Find a way to transform expression a into expression b. 
Reduce Goal: Reduce the difference d between expressions a and b. 
Apply Goal: Apply operator q to expression a. 
One method is associated with each of these goals. The method associated with 
the transform goal consists in (1) matching the two expressions to find a 
difference q between them; (2) setting up the reduce subgoal of reducing d— 
if that goal is attained, a new transformed expression a’ is obtained; and (3) 
Setting up a transform goal of getting from a' to b. The method associated 
with the reduce goal consists in (1) finding an operator q relevant to the 
difference d; (2) determining if the conditions are met for applying i | 
and (3) if so, applying the operator; if not, setting up the trains dem = 
$ reducing the difference between a and the conditions necessary or app ying 
1- These goals and methods are applied initially to developing a v 
Solving the problem; the plan is obtained by substituting for tie nina pe uu 
m a simplified problem from which some of the pai een ae 
the Simplified problem can be A the steps in its solution can 
“es to the solution of the original problem. . NE 
It is ute that this sort of general problem-solving pagum = 
ua theorems in logic; other uses for it are likely to develop as refinem 
It Conti 
Utpte-Fioyfun Programs. A particularly lively field in which aa 
problem-solving is being studied is the development of game-playing ines cd 
lt Is, of course, elementary to write a computer program which will play a 
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perfect game of tic-tac-toe. More complicated programs have been written 
for large computers which play excellent games of checkers (Samuel, 1959). 
But the interesting games are the more complicated ones, such as chess and go, 
for which optimal strategies are not known. A number of chess-playing pro- 
grams have been developed (Bernstein, 1958; Kister et al., 1957; Newell, 
1955; Newell, Shaw, and Simon, 1958; Shannon, 1950); no chess program 
has, in fact, yet been able to beat even a rank human amateur. It seems 
likely that the general principles included in such programs could be used to 
write a really expert program if only large enough machines were available. 
But it is doubtful if a world's-champion program could be written, no matter 
how large the computer, with present-day understanding of the game; the 
heuristic principles necessary for championship play are too imperfectly under- 
stood. There appears to be no current attempt to write a go-playing program. 
This is unfortunate, since go is a more elegant game than chess, in the sense 
that its rules are far simpler and more general; the difficulty of this extraor- 
dinarily difficult game arises Solely out of strategic and tactical issues that 
arise in the play, rather than out of idiosyncrasies of particular pieces. Thus a 


80 program might be a better simulation of human problem solving than a 
chess program. 


Applications for Artificial Intelligence 


Three varieties of research on applications of computer technology make 
extensive contributions to knowledge about artificial intelligence. One is the 
development of convenient programming la 
already about that (but see U. S. Dept. of C 


i as a result of research or any other information- 
gathering process, the problem of locating any Particular item of information 
becomes ever more difficult. Obviously a computer could serve very effectively 
as the agent for such retrieval, if only a Satisfactory cross-indexing and 
question-interpreting system could be found for it. The development of such 
à cross-indexing System requires solution of the problem of providing the 
computer with multiple-access long-term memory; the multiple-access feature 
IS crucial. Much Progress has been made, 
from solved (see McCarthy, 1958; Perry, 


problem is obvious. The deficiencies in co 


be remedied in order to make machine translation possible are pattern recog- 


nition, for the purpose of identifying phrases, as distinct from single words; 
multiple-access long-term memory, to perform Vocabulary look-up operations 
fo ases an for si s ; $ 

r phrases rather than for single words; and problem solving, to make it 


—Ward Edwards 
- n 


Possible to put together meaningful sentences. It is now possible to perfo: 
Na aces, kind a machine translation; the translation can be on € 
of the CEN translation by a man who need not know the language 
iginal. Steady progress in this area is to be expected. (See Browr 

1958; Locke and Boothe, 1955; Yngve, 1958.) nibii 
enr lae TUI, Hite of the IRE Transactions on Human Factors in 
fous a s ( : -HFE2, No. 1) is devoted to automation of human func- 

. contains fascinating review articles about list languages, man- 
Computer decision processes, speech recognition, and problem solving dinh 
others, And it contains a complete bibliography of the literature on artificial 
Intelligence, with a detailed topical index. It is therefore the indispensable 
tool for anyone who wishes to pursue further many of the topics in this 
chapter, especially those of the final section. 


CONCLUSION 


I think that I shall never see 
A calculator made like me. 
A me that likes Martinis dry 
And on the rocks, a little rye. 
A me that looks at girls and such, 
But mostly girls, and very much. 
A me that wears an overcoat 
And likes a risky anecdote. 
A me that taps a foot and grins 
Whenever Dixieland begins. 
They make computers for a fee, 
But only moms can make a me.” 
Hilbert Schenck, Jr. (1960) 
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HAVING SEEN IN PREVIOUS CHAPTERS SOME- 
thing about the kinds of things man can do, so long as 
his inputs and outputs are adequately provided for. we 
need now to look more closely at these two latter 
aspects of man's functioning. What are his sensory 
capabilities, that is, the characteristics of his input 
mechanisms? Similarly, what do we know about the 
nature of his output, his motor response capabilities? And for both these 
aspects of his behavioral make-up, what are the limits within which they 
operate, under normal conditions as well as in certain environmental ex- 
tremes? These are the questions with which this chapter deals. 

Considerations of man's sensory and motor capabilities and limitations 
enter importantly into decisions made in a very early stage of system develop- 
ment—in particular, those concerning the allocation of functions to man and 
machine. In participating in or contributing to such decisions, the psychologist 
has as a part of his repertoire a systematized body of knowledge called psycho- 
physics. There are precise methods by means of which psychophysical data 
are collected, and there is much detailed information, particularly about the 
functioning of the senses. Regardless of the amounts of these existing data, 
there is much that is not yet known, and the psychologist often finds he is 
required to conduct experimental measurements in order to answer some 
specific questions about system design. 

The authors of this cha 
Which data on human ca 


pter offer an account of some of the ways in 
pabilities are applied to the solution of system prob- 
lems. Realistic examples are also provided to make possible an understanding 
of the many alternatives and the successive decisions that enter into this 
process. A description of the relations of data on sensory limitations to system 
planning and design is followed by a discussion of human motor capabilities 
and their relevance to man-machine functioning, and finally by a treatment of 
the effects of environmental stresses on performance. 

. The problem of using information on man's sensory and motor limita- 
tions in the allocation of functions to men and machines is conceived as one 
of determining trade-off equations that predict system effectiveness. In the 
pursuit of this effort, the needs are seen to be for more psychophysical data, 
expressed in the probabilistic terms which will fit predictive formulas, as well 


as for rules of interpreting such data over broad ranges of variables. 


114 


HUMAN CAPABILITIES 
AND LIMITATIONS 


Joseph W. Wulfeck 


and Lawrence R. Zeitlin 


If man is proposed as a system component, an understanding of his 
unique sensory and motor capabilities and limitations is required. An appre- 
Ciation of the methods by which man's performance as a system component 
may be predicted during system design, assessed during system development, 
Or evaluated during system test is also necessary. Clearly, there are certain 
Steps in system development which can only be taken effectively by applying 
Principles that are uniquely psychological. At these points the engineer, mathe- 
Matician, or statistician is at a handicap without. the content of psychology; 
the medically or physiologically oriented worker is at a handicap without the 
methodology of psychology. . 

It is not the purpose of this chapter to presume to present a concise 
review of the literature on human capabilities and limitations. A bibliography 
Of the literature documenting human capabilities and limitations and of the 
Methodology for determining them would fill this book. It is the rather more 
humble purpose of this chapter to attempt to illustrate the significance of 
basic data on human sensory and motor capability, and the appropriate 
Manipulation of those data, in system design and development. Therefore, we 
are concerned primarily with defining the contribution to system development 
9f data normally obtained by psychophysical and psychophysiological meth- 
Ods, and secondarily with documenting the manner in which those data are 
Upically applied to system development. 
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of raw radar data to a human monitor who will perform the detection, identi- 
fication, and threat-evaluation functions. Given the characteristics of the radar 
and of the targets in question, how will the vigilance behavior of the human 
operator aflect system performance if the system requires that 90 percent of 
the targets which appear be detected within five seconds of their appearance? 
An entirely hypothetical conclusion might be that, if 90 percent of targets 
exceed the detection threshold within five seconds, 4 percent will fail to be 
detected for an additional three seconds after their appearance, simply be- 
cause of the operator's vigilance behavior in that situation. Therefore, 90 per- 
cent target detection will be achieved only after eight seconds of target appear- 
ance on the display, and such a level of detection will not quite meet the 
requirements of the system. This result has been reached by looking through 
the equipment at the Operator’s capabilities and by drawing a conclusion 
which relates to system effectiveness. Since the conclusion was negative, one 
may choose to look at the Operator’s capabilities through another piece of 
equipment—say, a longer-range radar—in order to find one which, when 
combined with the operator’s vigilance characteristics, will meet or exceed 
system requirements. If a longer-range radar would cost more than automatic 


target detection equipment, one might choose to reallocate the functions to 
man and machine. 


, Ambient Lighting. As an alternative to finding a different piece of 
equipment (the longest-range radar available may have been employed), one 
might instead look at the equipment through the broad matrix of the oper- 
ator’s capabilities. If there are requirements arising from cost, weight, size, 
or power restrictions to use that particular radar and display; and if, com- 
bined with the operator’s vigilance characteristics, it fails to meet system 
requirements; can the equipment be modified or the situation be designed 


SO as to meet System requirements by utilizing some other aspect of the 
operator's capability? An entirely hypothetical ans 
be that a special chromatic filt 


xis of the observer, 90 percent will then be detected within five seconds 
of appearance on the display —a positive conclusion that the system will meet 
its requirements. 


operator functions. ys 


, It is both significant and comforting to realize that the same kinds of 
ata are required to make both Sorts of decisions, However, it is also signifi- 
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cant and distressing to realize that the time pressures under which the two 
Sets of decisions must be made are very different, and that the magnitude of 
those pressures is usually inversely related to the importance of the decisions 
which must be made. It was pointed out earlier that decisions regarding the 
allocation of man-machine functions influence subsequent decisions through- 
Out the remainder of the system’s development. They are important indeed. 
However, it is precisely these decisions which must be made and revised under 


the greatest time pressures. 


Experimentation 


Occasionally, during the stage of man-machine function allocation, fre- 
quently during equipment design, and always in a basic research program, it 
should be possible to do the experimental work necessary to nail down a 
particular human capability in relation to the physical or physiological varia- 
bles which would be expected to affect that capability. The experimental work 
Which is part of a basic research program may be directed ultimately at an 
understanding of the psychophysical or psychophysiological implications of a 
particular stimulus-response process. But many other variables must be simul- 
taneously considered in any given system-design decision. For example, in a 
Tesponse indicating a visual detection or acuity threshold, parameters of the 
Stimulus, such as size, intensity, wave length, contrast with background, as 
Well as psychophysiological factors, such as state of adaptation and retinal 
area, may be directly relevant to a considerable variety of specific man- 
machine function allocation decisions. . 

Experimental work done during the man-machine function allocation 
Stage of system development will usually be designed to determine human 
Capability within ranges of physical variables in which gaps have been left 
in existing basic research data or in ranges of variables beyond those covered 
by existing data. Experimental work done in support of the equipment design 
Stage of system development will normally take the form of studies designed 
to discover optimum levels of particular physical variables in relation to the 


human function allocated. 


Use of Existing Data 


Time is usually not available to generate experimental data specific to 
the initial decisions allocating functions to man and machine, nor is formal 
€Xperimentation ordinarily possible as allocation revisions are required. Such 
initial decisions must, therefore, frequently be based on interpretations and 
extrapolations from existing data. Since those data will rarely be specific to 
the Situation, the process requires explicit generation of hypotheses but allows 
Only for implicit test of these hypotheses a priori and revision of them a pos- 
leriori based upon observational or, at best, cursory experimental data. Gen- 
erally, the hypotheses can only be stated in terms which in an experimental 
Situation would lend them to rejection by a one-tailed t-test. 
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As an example, let us consider another hypothetical system. Goals for 
the system are stated as usual in general terms: to develop a vehicle-borne 
antitank weapon system which will provide first-round kill probability of 
98 percent. Some specific system requirements are that: 


1. it be adaptable to all armored vehicles now operational or which will 
become operational during the next 15 years; 

2. it be a daylight system, with potential for adaptation to night-time use; 

3. it defend against tanks up to a range of 5000 yards and as close as 500 
yards; 

4. it destroy a target at maximum range within 23 seconds of sighting the 
target. 


Pure engineering considerations determine that the weapon will be a 
missile. Could the system be fully manual? Could it be fully automatic? If 
neither, what would be optimum allocation of functions between man and 
machine? Innocuous, straightforward questions! A partial and very fragmen- 
tary list of the information related to human capabilities required to answer 
them follows: 


1. How is daylight defined? What is the lowest "daylight" brightness under 
which the system must operate? 
Can the target be seen by the operator under all conditions? 


a. What visual angle does an enemy tank subtend today? What angle 
15 years from now? 


b. What is the reflectance of the tank? 


c. Against what sorts of backgrounds will it appear? What range of 
contrasts? 


2. 


3. Can it be tracked at maximum speed and minimum range? 
à. How fast can it travel today? How fast 15 years from now? 
b. How fast can it change direction and speed? 
4. Could the missile be seen under all conditions? 
a. What visual angle would it subtend at 5000 yards? 
b. Would it have a visible plume? 
c. What is its reflectance? 
d. What is its range of contrasts against expected tank background? 
5. Can it be tracked under all conditions? 
à. How fast does the missile accelerate? 
b. What is its maximum velocity? 


6. What is the relation between optical sight magnification and 
à. target detection and tracking; 
b. missile acquisition and tracking? 

7 


- What is the relation between optical Sight reticle design and 
a. target detection and tracking; 
b. missile acquisition and tracking? 
8. If the missile firing vehicle is standin 
are the effects of its vibration on 
a. target detection and tracking: 
b. missile acquisition and tracking? 


8 still with its engine idling, what 
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9. What are the effects of heat shimmer along the line-of-sight on 
a. target detection and tracking: 
b. missile acquisition and tracking? 
10. What are the effects of recoil-producing firing transients on 
a. target detection and tracking: 
b. missile acquisition and tracking? 
11. What are the effects of missile propellant ignition: glare and smoke on 
a. target detection and tracking: 
b. missile acquisition and tracking, especially at very low daylight 
brightness? 
What are the effects of maximum rates of own weapon mount slew- 
ing on 
a. target tracking; 
b. missile tracking; 
c. gunner's nystagmus? 


Answers to these questions require a considerable amount of the data 
available on human visual performance and manual tracking functions. They 
also require data on the complex interactions of a variety of physical variables 
on these functions. Most significantly, if time is not available for experimen- 
tation, they require extrapolations from available data to allow tentative 
hypotheses to be drawn regarding the effects of unexplored ranges of the 
physical variables on the human functions. For example, the effects of various 
degrees of heat shimmer on target detection and tracking are not yet known, 
especially throughout broad ranges of target size, rates of target movements, 
ambient and target brightness, and target range. For the present, crude 
approximation of these effects must be inferred from scanty and marginally 
related data on the effects of vibration, blurring of the contour gradient, and 
the like upon visual functions. We can conclude, then, that to answer ques- 
tions such as (1) can a human operator detect and track a target under the 

and (2) can a human operator 


conditions required for system performance, 
detect and track a missile under the conditions required for system perform- 


ance, man-machine function allocation must be based on: 

1. basic data on human capabilities; 

data on the interactions of many 

human capabilities: and 

3. extrapolation from the data to range 
actions not yet included in available data. 


physical and physiological variables on 


N 


s of variables or higher-order inter- 


The classical psychophysical methods and modern adaptations of them 
are the primary means by which it is possible to relate man to his environ- 
Ment in any truly quantitative fashion. If one disregards the polemics which 
have surrounded development of psychophysics as a set of techniques for 
relating mind and body and refuses to be involved in the question of reality 
or universality of the psychophysical law, one can accept the psychophysical 
methods as fundamental tools of psychological measurement. Without them 
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there would be precious few psychological principles to apply to system 
development. 

Since the psychophysical methods were designed specifically to identify 
and define the stimulus and to relate change in stimulus magnitude to changes 
in response magnitude, they make it possible to identify those aspects of the 
system environment which can be manipulated effectively to mate with man's 
capabilities and to attempt to quantify the effects of manipulating them on 
human performance as a system component. In certain cases, the psycho- 
physical methods also enable one to establish the limits of system effective- 
ness when those limits are dependent directly upon man's own limitations. 

All the psychophysical methods in use today (including, for example, 
attitude scales) are modifications or refinements of three so-called classical 
methods: the method of adjustment, the method of limits, and the method 
of constant stimuli. A clear understanding of the means by which they are 
applied, of the controls necessary to produce reliable data with them, and 
of the pitfalls inherent in their use is assumed, for in the remainder of the 
chapter we shall present considerable illustrative data obtained by them. 


MAN AS A SYSTEM COMPONENT 


The primary role of a human in a complex system is that of an informa- 
tion-receiving, processing, or transmitting element. Depending on the specific 
allocation of functions to man and machine, the human may be asked to sense 
à variety of stimuli, evaluate the implications of the stimuli in accordance 
with instructions, and produce a control output. If we draw a block diagram 
of a system, we often indicate the position of man by a little rectangle labeled 
"operator," through which information vital to the functioning of the system 
passes. The characteristics of this little box are of great interest to both 
psychologists and systems engineers. For psychologists, much of the interest 


lies in the fact that prediction of the characteristics of the box serves as an 
excellent validation tool against which to test th 


à m e findings of laboratory 
experiments. This is the “real world" : 


pedi d" which is spoken of so mysteriously in 
the classroom. For the Systems engineer, a detailed knowledge of the charac- 


teristics of the box are important for specification of the input and output 
devices necessary to incorporate it into the system. 

At this point, a fundamental problem in system design arises. Since the 
description of the information integral to a system which must be processed 
by man or machine usually comes in engincering language, a conversion must 
be made to the terminology of psychology in order fo enter the body of 
accumulated experimental data on human sensory and motor performance. 
It is at this point that one must compare the iuivideatonchserilta charac- 
teristics of the various sensory modalities with the characteristics of fe infor- 
mation to be displayed to the Operator; considering, of course. the unique 
demands of the situation and other sensory demands on the operato: It is at 
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this point also that one must specify the nature of the decisions to be made 
and the control outputs to be taken from the operator. 

For the moment, let us restrict ourselves to the information-input side 
of our operator box. We shall assume here that we have made a priori assign- 
ment of specific information-handling functions to the man. The information 
comes to the operator as a stimulus, specified as to content, spectral charac- 
teristics, dynamic range, and the like. How is this information to be presented? 
What sense modality is to be exploited? 

Engineering design has progressed to the point where one type of signal 
energy can be converted to another type of energy with little loss in efficiency. 
In a radar, for example, radio waves carrying information coded in a time 
ated, for the operator, into pulses of light coded in a spatial 
oses, he can convert the same information into a series of 
as an auditory display. It has been suggested that such 
ted to the operator by means of vibratory or tactical 
ct that in the not too distant future someone may seri- 
ously propose an olfactory display. Generally, however, vision or audition is 
the sense modality of choice. In Table 4.1 we have presented summary data 
for several senses. The rows of the table represent various parameters of sense 
modality response; the columns list the senses themselves. By scanning across 
à row, we can compare various human input channels along a single parame- 
ter. By scanning down à column, we can assess over-all capabilities of a given 
sense modality. As a consequence of the nonlinearity between stimulus and 


response, the interactions between the various cells in Table 4.1 may become 
more important in some situations than the data in the cells themselves. The 
psychologist experienced in system development work keeps such a table, or 
a reasonable facsimile, in his head. He unconsciously matches cell entries with 
developments in the art of information display, in terms of ard feasi- 
bility, cost, or general desirability. Of course there: is bind best method of pre- 
senting information to an operator, although there are preferred methods, 
which cover a variety of generalized situations. Often a change in sensory 
Modality is effective because it permits the man to Padre information which 
was previously ignored or discarded from a Sarena We will illustrate an 
example of this later in this chapter, 1n describing the a of a tadar in which 
the best display proved to be a pair of earphones rather than a cathode ray 


p beyond the information in Table je is possible to extract a 
limited miber of sets of data characteristic den sense cate chosen, 
which schematize much of the information about ipa wit hin the movant 
as well as the variables that affect these EE ERI y Mgr nee i i 
of data within each sensory and response 2 n hs n eu) MERDA de 
" ack and possibly evolve first-cut so ntong to a great number 
Possible to attack an esign problems which one encounters 
ch data in the following sections. 
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consideration. Considered together, they demonstrate the complex interactions 
among visual or auditory capacities. It should be mentioned that for exem- 
plary purposes single variable functions are often presented for visual capacity, 


while multivariable functions are frequently chosen to describe fundamental 
auditory capacities. 


Vision 


In providing illustrative data for the visual sense, we shall deal with: 
(1) brightness sensitivity, (2) brightness discrimination, (3) color discrimi- 
nation, (4) visual acuity, (5) dark adaptation. The basic curves to be pre- 
sented deal primarily with stimulus characteristics. However, the first basic 
curve, Figure 4.1, is simply an anatomical plot of the number of rods and 


—— CONES 
——- RODS 


o o ° 
40° 60° 80 
NASAL RETINA FOVEA’ TEMPORAL RETINA 
: ir ir 4.1. Basic Visual Curve 1: Rod-cone population curve. Density 
of rods and cones from nasal to temporal edge of the reti Aft i 
1949. Data from Osterberg, 1935.) TP UNDIS, 


e t e cone system is largely responsible for 
detail and color vision, while the rod system provides for detection of mini- 
l much can be inferred about visual performance by 
knowing the stimulated area of the retina. Brightness sensitivity is a func- 
tion of the amount of the radiant energy and the wave length of that energy. 
Basic curve number two, in Figure 4.2, illustrates the spectral sensitivity of 
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ic Visual Curve 2: Spectral sensitivity curve. Relative 


Figure 4.2 Basi 
function of wavelength. (After Wulfeck et al., 1958. 


sensitivity to light as a 
Data from Hecht and Williams, 1922.) 


the rods and cones. From cone vision to rod vision, the region of maximum 


sensitivity shifts towards the blue end of the spectrum. The change from cone 


to rod vision occurs as luminance is decreased. As the eye becomes adapted 
to dark, scnsitivity increases gradually and the peak wave length becomes 


shorter. Generally speaking, therefore, the rods require much less radiant 
energy for vision than do the cones. However, rod response is achromatic. 
The third basic curve, Figure 4.3, is the dark adaptation curve. It has 
been selected because it illustrates how sensitivity behaves as a function of 
time in the dark. The first segment begins with a very rapid decrease in 
threshold and levels off after about ten minutes in the dark at a value approxi- 
mating the cone threshold. The second segment of the curve begins in about 


ten minutes and, for practical purposes, levels off from 35 to 45 minutes 
after adaptation has begun. The final value approximates the rod threshold. 
Colors are not recognizable as an aspect of sensing represented in the second 
Portion of the curve. 


Basic curve number igure 4.4, presents the brightness contrast 


four, Fi 
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LUMINANCE 


INFERRED 
INSTANTANEOUS 
THRESHOLD 


ARITHMETIC 
MEAN 


THRESHOLD LUMINANCE 
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TIME IN DARK 


that can iust be seen 


judgments of brightness or cont 


Separate two stimuli, describe the form 
tive movements of sti 


es one type of visual acuity, minimum 
resolvable, as a function of background luminance, At zero degrees displace- 


ment on the retina, no acuity ted at background luminances 
less than minus 4.6 log units. However, foveal acuity increases very rapidly 


with background luminance up to one millilambert where the curve levels off. 
At both four and 30 d a, large objects can be discrimi- 
i 30 degrees, there is no further 


can be measu 
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Figure 4.4 Basic Visual Curve 4: Contrast discrimination curve. The 
smallest brightness contrast that can be seen as a function of luminance. 
(After Baker and Grether, 1954. Data from Blackwell.) 


improvement in acuity at luminance values above those where foveal vision 
begins, and minimum resolvable acuity never falls below 30 minutes. At four 
degrees, on the other hand, acuity improves with increasing luminance up to 
about one millilambert. The interaction among background luminance, con- 
trast, and visual acuity is summarized in Figure 4.6. 

Other visual capabilities of importance in system development are dis- 
tance judgment or depth perception, form discrimination, and temporal dis- 
crimination including flicker discrimination, but the functions which we have 
presented are sufficient for present purposes. The primary functions are 
deceptively simple; it must be emphasized that each simple function is pre- 
sented as a pedagogical device to illustrate the general form of the family of 
functions of which it is representative. Table 4.2 illustrates the interactions 


between variables and several visual functions. 7 
Throughout our five basic curves and all families of functions deriving 


r: in no case is human response linearly related 
to the stimulus. In no case is the nonlinear response function a simple one: 
it is dependent upon a variety of stimulus parameters. Note, however, that 
the general form of a function is retained through the effects of a wide range 
of different stimulus parameters. It is for that reason that we have chosen and 
emphasized the five basic curves. If their general forms of nonlinearity are 


from them, the principle is clea 
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Figure 4.5. Basic Visual Curve 5: Visual acuity curve. Visual angle of 
the smallest detail that can be discriminated as a function of background 
luminance. (After Wulfeck et al, 1958. Data from Mandelbaum and 
Rowland, 1944.) 


understood, and if the nature of the families which they represent remain 
familiar, a giant step can usually be taken in relating human capabilities and 
limitations to system requirements. 

No one would pretend that applying the five basic sets of data will solve 
all system problems which require information on human visual capability. 
It is entirely possible that applying the data will not completely solve any 
single problem. But applying these data will give better problem solutions 
than would be possible without them. Even more important, attempting to 
apply the data when appropriate to a particular system problem may identify 
those aspects of it for which data are inadequate or lacking and may thereby 
efficiently pinpoint critical areas for basic or applied research. Let us apply 
the “basic curves” to another system example. 

System goals: effective continental air defense. 


Among system requirements: a high-speed, high-altitude, daylight manned 
interceptor system. 
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the function of visual sea 


Answer: a qualified yes—if we can 
between the daylight sky and t 
of dark adaptation to detect ra 


tem performance. 


Our scope must be small, so t I 
hich presents green blips against a yellow-green 


first question becomes: what happens if we push 


Proposed uses a phosphor w 
background. A very likely 
the luminance of the scope f. 
lowing: 


From the curve of Figure 
sion, he wi 


Using central (foveal) Vi 
the blips on the scope face. 


5 7 10 14 20 
MINUTES OF VISUAL ANGLE 


nd luminance and contrast required for bars 
gles to be seen under daylight conditions. (After 


e function allocation: can the pilot be allocated the 


control radar and also 


rch of the daylight sky? 
be sure that the difference in brightness 


he radar display will not demand a period 
dar signals which would compromise sys- 


hat it can be centrally located. The scope 


ace up toward sky brightness? We find the fol- 


4.1, it is apparent that since the pilot will be 
Il be using only cone receptors to fixate 
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Table 4.2 


Variables that Must Be Kept Constant or Carefully Controlled when 
Measuring Some of the Principal Kinds of Visual Performance 
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Color | | | | | 
discrimination | X | x X (MV)'" x X X kd X | | X 


| I 
* Variable being measured 


(From Wulfeck e; al., 1958) 


From the curve of Figure 4.2, it appears that raising intensity is a better 
approach to the problem than changing color, since the cones are alrcady 
most sensitive in the yellow-green portion of the spectrum. 

From the curve of Figure 4.3, it can be seen that less time will be re- 
quired for adapting to the raised level of illumination. 


angle subtended by the smallest target blip and using the act 
of blip and background, we could use Figure 4.4 to fi 


nd whether the blip 
would be detected at the contrasts involved. However, a 


‘safety factor" should 
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be incorporated to make certain that contrast will be sufficient, even if the 
pilot is not fully adapted to scope illumination. 

From Figure 4.5, it is seen that acuity will probably be reduced as a 
function of the decrease in contrast that will occur when scope-face brightness 
is increased toward blip brightness. 

Actually, Figure 4.5 shows that acuity decreases as a function of de- 
creased background luminance. However, this curve is based on data for a 
dark object against a light background; as background luminance is decreased, 
contrast between object and background is also decreased, and it is obvious 
that the decrease in acuity is due to the change in contrast rather than to 
the absolute change in background luminance. In the case of bright radar blips 
against a dim background, increasing the luminance of the background 
reduces contrast, and therefore acuity. (If one requires exact information on 
Visual acuity for the radarscope, data should be consulted that give acuity as 
a function of contrast for the actual luminance values involved and for the 
yellow-green portion of the spectrum. ) 

From the foregoing, it can be seen that if the luminance of the scope 
face is increased, other variables will probably have to be manipulated so 
that the pilot can detect target blips and locate them accurately on the scope 
face. If the position of the scope cannot be changed, so that its size can be 
increased without pushing other instruments out farther, one might alter, for 
n, luminance of blip, or shape of blip. 

Spectral Composition. We have already seen that the eye is most sensi- 
tive to the yellow-green illumination of the scope face. However, contrast may 
be improved by changing the wave length of the phosphor or the scope illumi- 
nation or by changing the filter (if there is one) for optimum contrast. In 
other words, to make up for the decrease in brightness (achromatic) contrast, 
one can increase color (chromatic) contrast. Perhaps a change to yellow 


phosphor would be best, since visual acuity is best oul oe 
In view of the foregoing, one might better increase bright- 


ng the blip, so that it again stands out from the 
brightened scope face. However, limits are imposed by (D) E p tis 
phosphor, and (2) the diffusion of the image that eimi e ds a 
small luminous object is increased. At this point 1t ys y set nan recon- 
sider what the best background luminance would Bs z om rac E ope. 

Let us consider the following data: In o ra m if t s get must 
shift his eyes back and forth from the daylit sky poem x appe ae SÉ the 
Scope be as bright as the sky for the pilot to see 1t c nd The pon Is, NO; 
a surface up to 100 times less bright than a a € E Pe un 
Still be seen with little adaptation time. Since t » va mem O : aylit sky is 
about 2000 millilamberts, scope brightness S r aes od » m ae to 
20 m-L or higher to ensure visibility of the ama "eni re : arget dee s 

Now, how bright must the target signal be to be detected against a back- 


example, spectral compositio 


Luminance. 
ness contrast by brighten! 
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ground of 20 m-L? Figure 4.4 shows that as over-all luminance values in- 
crease, lower contrast ratios can be discriminated. Therefore, a signal of 
50 m-L should be more than bright enough to detect against a background 
of 20 m-L. 

Shape of Signal. In addition to changing color and brightness, one 
might make the blip easier to detect by designing the input to change its shape. 
One might also make it bigger, but the position of a large blip cannot be 
determined accurately. If it were desired to explore this approach, however, 
the data in Figure 4.6 indicate that the visibility of the blip at a given contrast 
could be increased by making it larger. 


Auditory Capabilities 


The auditory capacities we shall deal with are (1) absolute thresholds, 
(2) differential thresholds, (3) subjective measures of audition, (4) masking, 
and (5) perception of speech. 

The auditory stimulus consists of only one element, the rapid fluctuation 
of air pressure at the eardrum. No matter how complex a sound, it can be 
Tepresented as a variation in air pressure over time. This comparative sim- 
plicity of the auditory stimulus is a mixed blessing. On the one hand, it makes 
it possible to generate, transmit, and reproduce a wide variety of sounds with 
relative ease; on the other hand, it interferes with investigation of human 
response to single attributes of the auditory stimulus. It is difficult, for exam- 
ple, to separate the influence of frequency from intensity in a practical experi- 
ment. Suppose the attempt is made to measure the intensity diflerence limen 
by presenting a sound to a sub 
sity of the sound over 
change in intensity, 
detectable independen 


| as a base point the amount of energy 
necessary to just detect a sound at the 1000 Cps as zero decibels (db) (a 


sound pressure of .0002 dyne per square centimeter), then the amount of 
energy necessary to detect a sound at 20 cycles is approximately 10,000,000 
times greater, or 80 db above the base point. The human ear is actually some- 
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what more sensitive at frequencies higher than 1000 cps, the threshold of 
audibility being 5 to 10 db lower at about 3000 cps. Figure 4.7, our first basic 
curve in audition, shows the threshold of audibility plotted as a function of 
frequency. The upper threshold, the threshold of feeling, appears to be rela- 
tively independent of frequency and occurs at approximately 140 db across 
the audible range. 


189 AREA OF FEELING 


140 


120 


100 E 


“AREA OF HEARING: 
80 : an 


60 
40 


20 
AREA OF NO HEARING 


SOUND PRESSURE IN DB (RE 00002 DYNE/CMÊ) 


100 1000 10,000 
FREQUENCY IN CPS 


4.7. Basic Auditory Curve 1: The threshold of audibility as a 
Figure 4.7. 


function of frequency- 


Our second basic curve, Figure 4-8, shows the dier eed 
threshold as a function of the frequency and apnd. n s E get tone 
This figure is a three-dimensional surface, ind peo cm ier, m is 
difficult to manipulate single attributes oi ios s ice makes it exilis es 
lion of auditory space bya iuwediied e of either s bie 
consider simultaneously the implications T E ms can see from it = 
intensity upon perception of the test stimutus. gure 
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that the size of the difference limen (DL) for pure tones depends rather criti- 
cally upon the sensation level of the stimulus. In general, the DL is constant 
for sensation levels above 20 db, and is close to .5 db. 


Increment in db 


requency in cps 


s Figure 4.8. Basic Auditory Curve 2; Intensity discrimination curve. The 
differential intensi y threshold (DL) as a function of the frequency and in- 
tensity of the Stimulus tone. (After Licklider, 1951. Data from Riesz, 1928.) 
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Figure 4.9. Basic Auditory Curve 3: Frequency discrimination curve. 
The differential frequency threshold as a function of frequency and intensity 
of the stimulus. (After Licklider, 1951. Data from Shower and Biddulph 


1931.) 


that there is a correlation between intensity and loudness and between fre- 
quency and pitch, this correlation is by no means invariate. The loudness 
Versus intensity function is different for different frequencies. The pitch 
Versus frequency function is highly dependent upon loudness. Basic curve 
number four, Figure 4.10, is another three-dimensional surface, this time 
Showing loudness as a function of intensity and frequency. Loudness in this 
figure is represented in units of measurement called sones. (A sone is defined 
as a loudness of a reference tone of 1000 cps at 40 db sound pressure level. 
A sound whose loudness is 2 sones is twice as loud as one whose loudness is 
l sone, and so on [Rosenblith and Stevens, 1953]. It may be seen from the 
figure that the loudness of a tone increases rather slowly as the intensity of 
the stimulus is increased from the threshold of audibility to 60 or 70 db. The 
loudness then begins to increase quite rapidly as the intensity of the stimulus 
is increased. At 1000 cps, an increase in the intensity of the stimulus from 

ess of five sones. The same 


‘ in loudn 
Zero db to 40 db produces an increase 1n $ 
40 db increase in win: from 60 to 100 db, however, produces an increase 


In loudness of 65 sones. oe ‘ 
Masking. Acoustic stimuli are rarely presented in isolation. The 


Increment in cps 
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Figure 4.10. Basic Auditory Curve 4: Loudness as a function of in- 
tensity and frequency. (After Stevens and Davis, 1938.) 


desired stimulus must usually compete with other auditory stimuli, ambient 
noise, and transmission noise, all of which may interfere with perception of 
the desired tone or message. This phenomenon is known as masking. The 
percentage of masking is usually defined by determining the amount that the 
desired stimulus must be increased before it is perceived. It is obvious that 
the amount of interference between two stimuli will depend very much on 
what the stimuli are. Two pure tones will behave differently from a combina- 
tion of à pure tone with a noise. A number of experiments have been per- 
formed in which one pure tone masks another. The conclusions which can be 
drawn from these experiments are briefly as follows: (1) There is more 
masking between tones that lie close to each other in frequency than for tones 
which are farther apart. (2) A low frequency tone is more eflective in mask- 
ing a high frequency tone than vice versa. (Licklider, 1951.) 


Of greater concern to the system designer is the effect of noise on mask- 


ing of a desired stimulus. Figure 4.11 shows the effect of random noise (white 
noise) on the perception threshold of pure tones. The masking elect of such 
noise appears to be relatively constant across the frequency range 

Speech. Most communication to and from operators in systems func- 
tions is in the form of speech. We are often apt to overlook the fact that the 
operator’s earphones may be the most important display in the entire system. 
Some of the physical properties of speech are quite unlike those of the stimuli 


we have been considering. Perception of speech differs sufficiently from per- 
ception of other stimuli to justify separate treatment. 
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Figure 4.11. The masking effect of random (white) noise on the per- 
ception threshold of pure tones. The ordinate indicates the intensity re- 
quired for the pure tone to be just audible against random noise masking 
of the level shown 4s the parameter of each curve. (After Hawkins and 


Stevens, 1950.) 


The average speech power emitted by a speaker on a conversational level 
hly to a sound from 80 to 85 db. The 


is quite small and corresponds roug : 
distribution of speech power with frequency is reasonably constant for both 
male and female speakers. Figure 4.12 illustrates average speech spectra. 


In many applications, the presence of unwanted masking noise impairs 
the listener's ability to interpret all speech sounds correctly. The articulation 
Score, a quantitative measure of the efficiency of speech communications, 
represents the percentage of test words which can be transmitted correctly 
Over a communication channel. If the channel is nearly perfect, the articula- 
tion score is close to 100 percent. Figure 4.13 shows the articulation scores 
for three different types of test material. The test items were masked by white 
noise, and the percent items correct are plotted as a function of the signal-to- 
noise ratio in decibels. We can sce from this figure that the amount of masking 
is not only a function of the signal-to-noise ratio but also of the content of 
the material. The more familiar the material. the less it is masked. It appears 
that the contribution of human interpreting functions to speech reception is 
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Figure 4.12. Average speech spectra. (From Licklider and Miller, 
1951. After Dunn and White, 1940, and Rudmore et al., 1944.) 


quite marked. If we distort speech by passing it through a system that is 


selective in respect to frequency, as in the case of a high- or low-pass filter 
or a frequency-restricted carphone or microphone, the articulation score also 
changes. Figure 4.14 illustrates the effect on the articulation score for speech 
passed through high- or low-pass filters. From these curves we see that there 
is only a small increase in articulation score if we pass all frequencies above 
5000 cps. If we pass only those frequencies below 300 cps, the articula- 
tion score drops to zero. Likewise, a filter passing all frequencies above 
300 cps gives an articulation score close to 100 percent. The crossover point 
occurs at 2000 cps, hence a speech communication system passing either all 
frequencies below or above 2000 cps will yield equal results, providing about 
70 percent syllable intelligibility (French and Steinberg, 1947), Speech over 
such a system will not sound natural, but it will be intelligible. In general, 
however, a frequency range of from 600 to 3000 cps is considered as the 
minimum desirable for speech transmission. 


Specch can also be distorted considerably in 


amplitude before the articu- 
lation score drops to an unusable 


amount. In fact, deliberate amplitude dis- 
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Figure 4.13. Articulation scores for three different types of test ma- 


terials. (After Hudgins et al., 1947.) 


tortion or peak clipping is commonly encountered in speech communication 
devices as a means of using available transmitter power more efficiently. The 
number of decibels by which the amplitudes of the peaks are reduced by 
limiting defines the amount of peak clipping. Infinite peak clipping will reduce 
the speech to a succession of rectangular waves. Surprisingly enough, speech 
can be distorted to this extent and still retain an articulation score above 


70 per cent (Licklider, 1946). 


In general, we may say that speech reception is a human capability that 
g " 


has almost defied automatic or mechanical duplication. Humans can interpret 
speech distorted in frequency, amplitude, phase, and under conditions of high 
masking noise. Some of this capability lies in the analytic ability of the car, 
but by far the larger part lies in the nature of speech itself—a highly redundant 
communication process—and in the higher nervous centers of the listener. 
If the number of speech commands or messages IS limited, only a small frac- 
tion of the message must be perceived to enable the listener to reconstruct 


the message in its totality. 


As with data on vision, we have attempted to identify basic 


Example. 
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Figure 4.14. The effect on syllable articulation score of passing 


Speech through high-pass or low-pass filters. (After French and Steinberg, 
1947.) 


development follows. 


Radar engineers have been aware for some time that a radar return 
carries considerably more informati 


g e to moving parts (propellers and wings), 
changes in the refractive index of the air, and Doppler shifts due to the relative 
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many purposes the integration of return energy is valuable, however. Systems 
designers have recently become aware that the fine information on the radar 
return can aid in identification and evaluation of the gross aspects of the 
return. This is particularly true in the case of weather radar. 

A weather radar detects clouds by bouncing radiation off the rain drops 
within the cloud. The radar return gives position and general size, but it does 
not provide significant information about conditions within the cloud. By 
recovering fine-grain information from the radar return, it is possible to assess 
turbulence within the cloud by observing the relative motion of areas of rain- 
laden air. How is this material to be presented to the operator? The video 
display is already being used to give size, distance, and azimuth; the response 
rate of the eye is also limited. It has been suggested that the material be 
extracted from the video signal and presented to the operator through ear- 
phones. Turbulence modulation of the radar signal is relatively rapid, although 
somewhat below the audio range. Thus, it cannot be presented to the operator 
directly. It can, however, be used to modulate another tone well within the 
operators range of hearing. The problem we are now concerned with is the 
location of this tone in the auditory spectrum and the type of modulation to 
be impressed upon it. One must ask the following questions: 

1. What frequency should the carrier tone have? 

2. Should it be frequency or amplitude modulated; that is, should it change 

in frequency with a change in turbulence, or change in intensity? 

3. How loud must the carrier tone be against a noise background of at least 

60 db? 

4. Will it interfere with speech perception? 

From Figure 4.7, it may be seen that the desirable range of frequencies 
for the carrier tone is between 400 and 4000 cps. If a tone outside these 
limits is chosen, the amount of power necessary to present the display may be 
excessive. T : 

From Figure 4.8, it is evident that it takes a change in intensity of at least 
1 db for a tone 30 db above the threshold before an observer can detect the 
change. This increment drops to about .5 db at 60 db above threshold. This 
gives an amplitude sensitivity of from 10 to 30 percent of the stimulus power, 
depending upon the sensation level chosen. " em 

From Figure 4.9 one can see that the DL for requency modulation is 
somewhat smaller in terms of percentage of variation of the test signal at 
reasonable sensation levels. Above 30 db and at 1000 cps a .3 percent change 


can be detected. 


On the basis of the information presented so far, it is difficult to make a 


decision between frequency and amplitude modulation. It is known, how- 
ever, that pitch stability tends to be greater than loudness stability over a 
Period of time. Consequently, a frequency-modulated toue-would be preferred. 
More important, if the carrier tone is to be presented at various levels of 
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amplitude, a given percentage change in intensity will not be perceived as a 
constant percentage change in loudness (Figure 4.10). 

From Figure 4.11, it is apparent that a pure tone to be detected against 
a random noise background of 60 db must have an intensity of at least 80 db. 
If amplitude modulation is used, extreme changes in the carrier intensity level 
may force the signal, on the one hand, into a near painful area, and on the 
other hand into an inaudible region. If frequency modulation is chosen, the 
tone can be made comfortably discernible against the random noise back- 
ground and held at approximately the same amplitude. 

What should the carrier frequency be? Will it interfere with speech per- 
ception? These questions are intimately related. From Figure 4.14, one sees 
that a low-pass communication system which cuts off at 3000 cps permits at 
least 80 percent syllable articulation. A communications system with syllable 
articulation score of 80 percent will transmit words in sentences virtually 
unimpaired. If the carrier frequency is positioned at 4000 cps, permitting 
deviations of = 1000 cps, it may be possible to present both speech and the 
auditory display on the same communication channel without too much 
mutual interference. At the same time, it can be seen from Figure 4-7 that 
this is still a rather sensitive area in the auditory spectrum. 

Conclusion: a frequency modulated carrier tone of 4000 cps, at least 
80 db above threshold, meets the stated requirements. In an actual system it 
would probably be undesirable to stop the design effort here. Of necessity, 
other factors would be considered, including the pulse repetition rate of the 
radar, the characteristics of the audio system, the characteristics of the ear- 
phone, and the nature of the ambient noise. 


Other Senses 


The contribution of information about system operation obtained from 
other sense modalities may be extremely important to system performance. 
Noxious olfactory stimulants are often incorporated in liquid and gaseous 
materials to serve as warning indicators of leaky hydraulic or pneumatic sys- 
tems. Gustatory sensitivity identifies a spoiled steamed clam long before it 
becomes a dangerous agent in the human digestive system. Body heating may 
indicate a faulty air conditioning system. Indirect sensory effects may identify 
a malfunctioning oxygen-breathing system. 

Excessive vibration, stimulating tactile and kinesthetic receptors, ordi- 
narily indicates a malfunction in any mode of transportation. Vestibular sensi- 
tivity may provide the only sensory inputs useful for orientation in certain 
kinds of system operations conducted at night. Overheating commonly indi- 
cates malfunction in electronic and some mechanical equipment. 

However, with the exception of shape-coded controls and skin “buzzers” 
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displaying information to tactile and kinesthetic receptors, the senses other 
than vision and audition have not been called upon in system development as 
candidates in the man-machine function allocation process. Their significance 
in system development to date has fallen mainly in the area of equipment 
design for comfortable human operation, in optimizing performance through 
controlling the operating environment, and in determining the requirements 
for radical system environments, such as those necessarily encountered in sub- 
marine, high-altitude, underground, or completely closed ecological systems. 
In most cases basic sensory data, such as absolute or discrimination thresh- 
olds, have little significance, for the emphasis is on meeting environmental 
requirements for maximum work efficiency and minimum fatigue. For those 
reasons we will omit presentation of basic data for senses other than vision 
and audition, with the realization that the data do exist and can be organized 
efficiently for application to system development if necessary. 


=] MAN AS AN ELEMENT IN CONTROL SYSTEMS 


For every manned position in a man-machine control system provision 
must be made to permit the operator to return processed information to the 
system. The output device may be as simple asa pencil and pad of paper, 
or as complex as the fully instrumented cockpit of a high-performance air- 
craft. In many cases, the operator's function is to control some system 
parameter to agree closely with a command function derived from the external 
environment. The general control operation comes under the rather broad 
general category of “tracking”; however, the task may vary from a simple 
trapping of a blip with a light gun on a radar screen to driving an automobile 
at 60 miles an hour on an ice-covered expressway during rush hour. In effect, 
the operator is performing as a servo link whose objective is to reduce system 
error in following the desired forcing function to a tolerable quantity. What 
differentiates the human operator from an electrical or mechanical servo link 
is the ability of man to modify his own characteristics to match the require- 


ments of the controlled situation. 


Transfer Functions 

en human performance as an element in a control 
system and servomechanism theory has prompted much research into the 
Specification of transfer functions which would describe the nature of human 
Tesponse for specified inputs and outputs in the same manner as the response 
of a servo system. Unfortunately, from the point of view of theory, the range 
Of variables which importantly affect human performance in control systems 


The analogy betwe' 
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is far greater than the range handled by servomechanism theory. Among these 
variables are the mode of control, the kind of tracking, display, and control 
relationships, and the nature of the forcing function itself. The interaction 
between motor performance and sensory performance is extremely great. 
Transfer functions developed from a simple application of servomechanism 
theory in human performance could only apply if the human operator's 
responses were completely or very nearly linear. However, we have seen in 
earlier portions of this chapter that human responses to stimuli occur in a 
generally nonlinear fashion. 

If one chooses to restrict the scope of investigation to information pre- 
sented along a single continuum of a sensory modality and measures a control 
response in the absence of complex system dynamics, human transfer func- 
tions can be developed which are reasonably accurate for certain types of 
control tasks. According to Goode and Machol (1957), three sets of per- 
formance criteria are applicable to servomechanism design. These are tran- 
sient response, steady-state conditions, and stability. Transient response refers 
to the manner in which a device settles down to steady performance after an 
abrupt change in the input stimulus. Steady-state conditions refer to the error 
between input and response which remains when all transients have died out. 
Stability, of course, refers to the consistency of performance of the system to a 
variety of inputs. In this section, we shall use these three criteria to describe 
human capabilities in a control system. 


Human Responses in a Control System 


In most studies investigating human transient response, a stimulus moves 
sharply in one direction and the operator is required to track the position of 
the stimulus with the controlled element. Human response in this type of input 
occurs in three phases. During the first phase there is no appreciable motion 
at all. This is the reaction-time delay. At the end of this dead period, the 
subject moves the controlled element rapidly toward the position of the stim- 
ulus, ending with a small error. During the final portion of the response, the 
operator moves the controlled element slowly toward the final position of the 
stimulus, eliminating the remaining error as much as possible. The average 
duration of the reaction-time delay is about .25 second and generally agrees 
with other reaction times recorded to simple visual stimuli (McRuer and 
Krendel, 1957). The dynamic portion of the response, at least the first con- 
trolled movement, is generally ballistic, that is, the operator forcefully moves 
the control element by a single muscle impulse. It makes relatively little dif- 
ference how far he must move the control element. The time taken for simple 
ballistic responses is generally the same regardless of the extent of motion. 
This time is about .1 second. The operator determines the amount of error 
between the position of the controlled element and the stimulus, then grad- 
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ually reduces this error. Depending on the dynamics of the situation, he may 
overshoot the stimulus position once or twice before the final position. The 
total time taken to track a step function is usually .4 to .5 second. 

If the stimulus is not moved in steps but is moved continuously, one can 
measure the operator’s steady-state response characteristics. If the results are 
to be generalized to a wide variety of input conditions, it is best to use an 
input stimulus which appears random or nonpredictable in nature. Low- 
frequency white noise has often been used to control stimuli in tracking 
experiments. The results of studies using this type of input show that the 
highest input frequency which a man can track successfully is somewhat less 
than one cycle per second. Tracking accuracy drops off severely when the 
input frequency exceeds 2 cycles a second (Elkind, 1956). This figure com- 
pares well with the transient response of the human operator indicated within 
aragraphs. Since a random input stimulus is nonpredictable, 
the operator may be considered to be tracking a series of step functions of 
arbitrary amplitude and duration. Table 4.3 shows what happens to the 
correlation between input and output of a human operator as the frequency 


of the input becomes higher. 


the previous p 


Table 4.3 


Tracking Capability of Human Operator 


Forcing function bandwith * Average linear correlation 

(cps)" 

0.995 

.16 ji 

: F 0.99 

0.40 0.995 

064 0.98 

0.92 

"em 0.75 

ia 0.58 


" Random-appearing function made up of 40-120 sinusoids giving any desirable rec- 


tangular spectra. = 
? Passband from 0 cps to upper limit. 


(After Elkind, 1956) 
asic characteristics of human visual- 
Manual responses somewhat as follows: (1) The human RPEretot 18 funda- 
Mentally a low-pass device. Considered as an amplifier, his gain approaches 
Zero at high frequencies. (2) For low bandwidth inputs, his low-frequency 
gain can = very high but is finite, because of limitations of visual, motor, and 
Memory systems B) All performance in tracking unpredictable complex 
stimuli must cortain a delay analogous to the reaction-time delay or stimulus- 
Tesponse latency of the human operator in à discrete tracking task. 


Elkind (1956) summarizes the b 
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If the stimulus moves in a completely predictable manner, the operator 
will learn the control motions which must be made to minimize his error. A 
low-frequency sine wave is a stimulus of this sort. Three types of response 
occur to this form of stimulus. The first is “nonsynchronous,” in which the 
operator lags behind the motion of the stimulus. As the operator learns to 
anticipate the stimulus movements, his response changes to a "synchronous" 
mode, in which there is no lag. At this point he may be said to have learned 
the response. The mode of operation can be maintained for long periods of 
time provided the stimulus frequency is less than 3 cps. If the frequency of 
motion of the stimulus is much faster, the operator tends to drift out of 
synchronism and, for a time, his actions may serve to increase error rather 
than decrease it (McRuer and Krendel, 1957). The control system then be- 
comes unstable. If this occurs when piloting an airplane through buffeting, or 
driving a car down a winding road, the effect, of course, can be disastrous. 

In summarizing human performance as a control element in a servo sys- 
tem, we can state that man has a transient response of .5 second, can track 
forcing functions with high accuracy provided the input does not contain 
frequencies above 2 cps, and can track predictable, repetitive phenomena if 
the frequency is not much higher than 3 cps. At frequencies higher than 3 cps, 
instability of response occurs, the operator sporadically getting out of phase 
with the input, and error may be increased. 


ENVIRONMENT 


In the design of complex systems, man must be considered as an element 
which operates best in a fairly narrow range of environmental conditions. As 
mentioned in previous sections, environmental factors have a direct bearing 
on operator performance. Their effect on performance can be exerted in two 
ways. First, the environment may be such as to degrade a sensory modality 
directly. For example, visual performance suffers if the environment does 
not provide adequate illumination; auditory performance suffers in an environ- 
ment containing high ambient noise levels. Second, the environment may 
introduce physiological stresses which indirectly affect sensory or motor per- 
formance. Temperature, atmospheric contamination, inadequate oxygen, and 
radiation fall in this category. To some extent, a single environmental con- 
dition may operate both directly and indirectly upon human performance. 
Noise and vibration fall in this category. Of course, in addition to their influ- 
ence on operative performance, severe deviations from an environmental norm 
can cause physical injury or endanger life. The magnitude of the absolute 
limit of environmental stress is, roughly, inversely proportionate to the 
amount of time the stress must be endured. The absolute limits of human 
tolerance greatly exceed normal expectations, provided the time of exposure 
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is extremely short. For example, a properly clothed man can exist for one 
hour without sign of injury at temperatures ranging from —60° F to over 
140° F. For shorter periods of time, this range can be extended from —100° F 
to over 200° F. A man in a supine position can survive transverse accelera- 
tions of over 20 G for periods of 30 seconds and over 50 G for periods of 
1 second (Eckenrode and Abbot, 1959). 

For the most part, however, interest centers on a much smaller environ- 
mental range. One must assume, in systems design work, that equipment 
designers will not provide modes of system operation which subject an opera- 
tor to environmental stresses greater than his tolerance limits. For the purpose 
of illustration, we will consider environmental stresses in three areas: tempera- 
ture, acceleration, and noise. 

Temperature. Temperature is a stress which does not interfere directly 
with a sensory or motor mechanism, but influences the general comfort and 
Psychological functioning of an individual. The temperature of the human 
body depends upon the balance of heat produced and lost by the body. Under 
normal conditions, the body adapts to changes in temperature, humidity, and 
air movement, regulating the sweat glands and the rate of flow of blood in 
the skin to maintain the body temperature at about 98.6° F. As long as the 
individual can readily adapt to external changes, heat equilibrium is main- 
tained and the body feels comfortable. There is a slight physiological penalty 
which must be paid for this maintenance of equilibrium, however. If the 
ambient air temperature rises to much greater than 80^ F, there is some 
slight impairment in performance. By the time the ambient rises to 90° F, 
there is a demonstrable impairment in man's mental and physical processes. 
If one does not consider extreme temperature ranges, the effect of tempera- 
is slight for simple motor and sensory tasks, but in- 
ity of the task increases. Table 4.4 indicates the critical 
rformance impairment may be demonstrated. 

Acceleration. One important use of a human in systems operation is as 
a vehicle controller. In this kind of task, the operator may be subject to 
accelerative forces with every change in vehicle velocity or orientation. These 
forces must be controlled so as not to become too great an influence on per- 
ception of pertinent systems information and the performance of necessary 
tasks. It goes without saying that men function best at an acceleration of 16 
directed in a head-to-foot direction. If the acceleration in this direction in- 
Creases, sensory degradation occurs, due to drainage of blood from the brain. 
Vision is lost first, followed by loss of hearing, and then loss of consciousness. 
The “blackout” phenomenon generally starts at 3 G's acceleration but can 
be postponed by the use of appropriate clothing. If the acceleration is directed 
in the opposite direction, from foot to head, sensory performance is also 
impaired. 

The human is much less sensit 


ture on performance 
creases as the complex 
effective temperatures at which pe 


ive to acceleration in a transverse direc- 
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tion. Sensory performance is generally not affected unless the acceleration 
reaches 10 G or greater. In this case, there may be some blurring of vision 
because of the distortion of the eye. 

The most important effect of acceleration on human performance, after 
sensory degradation, is the restriction of gross body movements. Regardless 
of direction of acceleration, a man cannot move about freely at greater than 
2 G. The arms and the hands become difficult to move freely at greater than 
3 G. These limits of perception and motor performance determine the limits 
of operator participation in the control of high-performance vehicular systems. 

Noise. Noise is one of those environmental parameters which interferes 
directly with a sensory function and indirectly with over-all human perform- 
ance. As mentioned previously, noise directly affects speech perception, and 
consequently can influence system functioning to a great extent. However, in 
addition to masking speech, it is possible for noise to have an effect on other 
kinds of human behavior, particularly performance of tasks requiring con- 


Table 4.4 


The Critical Effective Temperatures at Which Impairment May Be 
Demonstrated, According to Various Sources 


Temperature (°F) 


Name and type of test Investigator Max. at which Demonstrable 
performance impairment 
remains normal 
Typewriter code (scrambled letters) Viteles 80 87 
Morse code reception Mackworth 87.5 92° 
Locations (spatial relations code) Viteles 80 87 
Block coding (problem solving) Mackworth 83 87.5* 
Mental multiplication (problems) Viteles 80 87 
Number checking (error detection) Viteles 80 87* 
Visual attention (clock test) Mackworth 79 87:55 
Pursuit (visual maze) Viteles 80 87 
Reaction time (simple response) Forlano 93 s — 
Discrimeter (complex response) Viteles 80 87 
Lathe (hand coordination) Viteles 80 87° 
Pursuitmeter Mackworth 87.5 92° 
Motor coordination Weiner 64.5* 91* 
Ergograph (weight pulling) Mackworth Bit eas 
Bicycle ergometer (heavy work) Liberson 64.5* 91.5* 
Weight lifting (heavy work) N. Y. Ventil. 64.5* 70* 
Comm. 


* Deterioration statistically significant. 

° Provided wet bulb does not exceed 86°F. 

* Effective temperature estimated from data in report. 
* Midpoint of a range of conditions. 


(After Eckenrode and Abbot, 1959) 
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centrated attention. For the purpose of discussion, it is convenient to divide 
behavior into mental and manual tasks and to divide noise levels into intense 
(above 100 db) and low-level noise. Over short periods, less than two hours, 
there is no clearly demonstrated effect of low-level noise on human perform- 
ance. If the time of exposure is extended, it may be possible to discern a slight 
degradation of performance for manual tasks. 

Intense noise generally has an effect on performance. The effect on man- 
ual tasks is usually small, the effect on mental tasks is somewhat greater. If 
the noise is very intense, ear damage can result, which can influence system 
functioning long after cessation of the noise. Table 4.5 lists the effects of 
exposure to noise on human performance (Eckenrode and Abbot, 1959). 


Table 4.5 


Some Effects of Exposure to Noise as Compared to Quiet on 
Human Performance and Other Processes 


Type of Noise level Noise duration Quiet level Effect of noise 
performance (db) (db) 

Addition problems 50 Continuous Not given No difference in 
number of correct 
solutions. Considerable 
increase in energy 
expenditure under noise 
as compared to quiet, 
especially during first 
few days. 

Continuous 120 Intermittent Ao s No effect 

acking and random 
ee 120 12 x 2 min. *  * Performance improved 
in 4 hours 
130 3 min. at *  * Performance improved 
middle and 
end of 4 
hours 

Stereoscopic 120 3 minutes * ^" No effect 

ranging 

Inserting pegs high Intermittent *  *  [nitial performance 

in pegboard clicks and slowed but over-all 

complex performance showed 
noise no difference 

Tracking requiring 115 continuous 90 Reactions in noise 

hand, foit; and eye 5.4% slower 

coordination 

Card sorting 115 continuous 90 No effect 

115 continuous 90 No effect 


Marksmankship 


Joystick pursuit 115 
tracking 


continuous 90 No effect 
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Some Effects of Exposure to Noise as Compared to Quiet on 
Human Performance and Other Processes 


Be nice S Noise duration NU Effect of noise 
Hand or foot 115 continuous 90 No effect 
key-pressing 
Key pressing to 120 10 minutes notgiven Time required initially 
translate letters longer; greater tension 
to numbers in noise 
Monitoring clock 114 last 11^ 83 Significantly poorer 
for erratic hand hours of 2- in last V? hour 
movements hour trial 
Conversation 0-60 continuous — Normal 

60-80 b — Raised voice 
80-100 us — Very difficult 
100-115 Ls — Shouting 
«115 “a — Impossible 
Comfort level 0-60 Continuous — Quiet and very 
in aircraft comfortable 
60-80 ^r — Comfortable 
80-90 x — Acceptable 
90-100 id — Noisy 
100-115 * — Very noisy and 
disagreeable 
115-125 be — Uncomfortable 
« 125 5 — Painful 


(After Eckenrode and Abbot, 1959) 


SOME GENERAL IMPLICATIONS 


This chapter has emphasized the position that psychophysical and psy- 
chophysiological information on human capabilities and limitations is most 
significant to system development when applied during the process of allo- 
cating functions to man and machine. It has attempted to show why the 
information becomes significant, what kind of information is available, and 
how the information that is available must normally be applied. It is time now 
to take a close, hard look at the implications of the position taken and of the 
material presented for past and especially for future system development. 

As indicated in Chapter 2 and reiterated here, the decisions made during 
man-machine function allocation establish the requirements for equipment 
design, work-place layout and environmental control. They determine man- 
ning, selection, training, and organizational requirements. Finally, they define 
to a large extent the nature of system test and evaluation. On the face of it, 
the position has such validity that we may well accept it as a psychological 
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principle of system development. But how can such a clear-cut principle have 
been so difficult to establish and so marginal in application? The answer is a 
painful one; Jordan (1961) makes the point nicely. He begins with a state- 
ment from Swain and Wohl (1961), to the effect that there is no adequate 
systematic methodology in existence for allocating functions between man and 
machine. This lack, Swain and Wohl suggest, is probably the central problem 
in human factors engineering today. They point out that ten years of research 
and experience have failed to bring us any closer to the goal than did the 
landmark article by Fitts (1951). Jordan then questions the utility of the 
“Fitts lists," which instruct us to compare man to the machine and choose the 
one who fits a function best. He concludes that if men and machines are not 
comparable, it is not surprising that the attempt to compare them proves 
fruitless. 

Jordan summarizes Craik's (1948) arguments, which recommend that 
human functions be described in terms comparable to the terms used in de- 
scribing mechanical functions, and concludes that whenever human function 
can be reduced to a mathematical formula, it is generally possible to build a 
machine that can do it more efficiently than a man; therefore, to the extent 
that man is comparable to a machine, he can be replaced by a machine. 
Birmingham and Taylor’s conclusion (1954) that, speaking mathematically, 
man is best when doing least, is extended to a non sequitur—if man is best 
when he does least, the best he can do is nothing. Clearly the rules for making 
man-machine function allocations have not been agreed upon. 

An attempt to seck an answer to this problem can be made with the 
assumption that man-machine function allocation decisions may be made as a 
series of trade-offs which predict desired operational effectiveness for the sys- 
tem. Those trade-offs must include data on human capabilities and limita- 
tions, but they also require data on machine and human size, weight, relia- 
bility, availability, maintainability and cost (in the human case, the cost of 
training, of environmental support, and the like). The trade-offs can be made 
properly in terms of cost or value: what will be the m he desired system 
operational effectiveness of using the limited-reliabi ity but ers repro- 
grammed human being, rather than the reliable, inflexible machine? Applying 
the techniques for making such trade-offs a priori, and predicting their cost, 

hich demands that psychophysical and psy- 


is therefore a growing necessity w. r 
chophysiological data as well as standard design data be evaluated by system 
analysts and operations researchers. Some psychologists have participated in 


application of these techniques to man-machine function allocation and sys- 
tem-design feasibility studies, and they have sometimes been criticized for 
being macroanalytic when microanalytic techniques are required to deal with 


the human. 


To date psychologists have been and continue to be somewhat limited in 


init ign i f participation and the sig- 
the initial stages of system design in the extent of p: l 
nificance of oA because of the form in which psychophysical, psy- 
chophysiological, and more general behavioral data are ordinarily available. 
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This is not an epistomological problem. It is not solved when one calls stimuli 
"inputs" and responses "outputs," or when one relates stimulus and response 
in psychophysical terms and concludes that the human may be considered 
an amplifier or some other electrical or electronic component. The problem 
is uniquely psychological, and it appears to be fundamentally a problem in 
psychophysics. 

The most stringent demands on the form of psychophysical and psycho- 
physiological data occur in the development of mathematical models for sys- 
tem simulation, cost analyses, or value studies. Probablistic data in the form 
of error functions are required for single-stage analytic probability models and 
for numerical probability or Monte Carlo models used in complex simulations. 
The only general psychophysical method which produces sensory data in that 
form is the method of constant stimuli. However, the method of adjustment 
and the method of limits have been used so extensively that a tremendous 
amount of the data available on human capabilities and limitations have been 
obtained by them. Unfortunately, variability indices obtained with those 
methods have not been, and are not normally, reported in sufficient detail to 
allow the statistical manipulations necessary for deriving probabilistic con- 
clusions from them. Furthermore, the method of constant stimuli is the most 
laborious and time-consuming to apply when gathering new data; the method 
of adjustment is much simpler to instrument, and the method of limits is 
much faster to use. These differences are of considerable significance within 
the operational framework, for when the opportunity for experimentation 
presents itself, the techniques used must be quick, easy, and efficient. Tanner 
and his associates (1956) have been active in adapting the constant stimuli 
method to operational problems, but much progress remains to be made. New 
psychophysical methods or efficient modifications of the old ones appear to be 
required. More needs to be known also about the rules for inferring the rela- 
tions among sets of psychophysical data and for relating psychophysical data 
across different modalities. These rules depend ultimately upon principles of 
psychological scaling. In addition, psychophysical scaling problems are rela- 
tively simple compared to the problems which must be solved if one is to 
scale, and therefore be able to trade off, in areas of complex decision making, 
such as threat evaluation or troubleshooting. 

Finally, knowledge is needed of the rules for interpreting psychophysical 
data obtained over broad ranges of variables, and of methods of designing 
psychophysical experiments which will give maximum interpolation accuracy 
through the widest ranges of variables with minimum administrative and com- 
putational difficulty. 

These requirements are severe and will not be met easily. However, they 
are requirements which must be realized as rapidly as possible if the mass of 
psychophysical and psychophysiological data describing human capability and 


limitations is to be uniformly useful in establishing the psychological princi- 
ples of system development. 
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BEGINNING AT THE DESIGN PHASE OF SYS- 
tem development, a series of critical decisions is made 
which have profound effects upon the parts to be played 
in the system by human operators, as well as upon the 
effectiveness of their performance. These decisions are 
based upon a determination of the nature of human 
tasks required by the system and have an effect upon 
the design and configuration of equipment. Psychologists customarily partici- 
pate in such design decisions, functioning as members of a design team, as 
consultants to designers, or as independent investigators of a design-relevant 
problem in human performance. The body of technology which is drawn upon 
for this effort belongs to the discipline called “human engineering" or “engi- 
neering psychology.” 

The principles of design available to the engineering psychologist have 
their fundamental basis in the psychologist’s conceptions of behavior func- 
tions, as well as in a considerable amount of data on human capacities and 
limitations. Beyond this, as shown in this chapter, a most productive approach 
to the design of human tasks has been the identification of human errors and 
the search for means to eliminate them. 

The elimination of human error requires first of all that the designer 
attain a clear conception of the nature of human tasks and the varieties of 
human functions embodied in them. Armed with this knowledge of what the 
human operator is “really doing” in interaction with a machine, it is then 
possible to draw the necessary conclusions concerning what his inputs and 
outputs should be. These in turn lead to a consideration of those particular 
features of equipment that link the man and the machine: displays and con- 
trols. For the engineering psychologist, then, this aspect of design is not simply 
a matter of “knobs and dials.” Instead, it is a question of designing links 
between the man and the machine which will permit optimal human func- 
tioning in the kinds of tasks to be performed by the human operator and, 


at the same time, efficient attainment of system goals by the man-machine 
combination. 


Some examples of what is meant by designing equipment for human use 
are described and discussed in this chapter, together with some empirically 
derived principles which pertain to information-processing and decision-mak- 
ing tasks. These are related to the problem of reducing human error in sys- 


tems, to some of the probable causes of such error, and to the need for a 
theory of human performance. 
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HUMAN TASKS AND 
EQUIPMENT DESIGN" 


J. S. Kidd 


Systems exist to facilitate man's control over his environment, or, ex- 
pressed in another way, to aid him in achieving an adjustive relationship with 
his environment. A system is best understood in terms of its intended use: 
the goals or purposes for which it was contrived. The instrumental aspect of 
the system in combination with its goal or purpose constitute a process whose 
primary characteristic is the elimination of a discrepancy between a pre- 
determined, intended state of affairs and an existing state of affairs. When 
such a discrepancy exists, the system is active; when it has been eliminated, 
the system is quiescent but retains the potential for action. A system is one 
of the major means by which man interacts with his environment in accom- 
plishing his purposes. 


Psychologists have good reason to be concerned with system develop- 


ment. In the broadest sense, anything that influences man's behavior—his 
interaction with his environment—is psychologically significant. For the pur- 
pose of this chapter, however, the subject-matter coverage may be substan- 
tially narrowed. The intent of our discussion is simply how to achieve effec- 
tiveness: how can one design and develop complex systems in such a manner 
that their intended purpose can be achieved most effectively? Since systems 
are used by and for people, how can our knowledge of human characteristics 
be introduced into the system design and development effort so as to derive 


the maximum human benefits? 


* The preparation of this chapter was in part supported by Air Force funds under 
contracts Nos. AF 33(616)-6166 and AF 33(616)—7122. Permission is hereby granted 
for reproduction in part or whole, by or for agencies of the United States Government. 
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On grounds of both logic and experience, psychologists have come to 
believe that they have a distinct contribution to make to what appears to be, 
fundamentally, an engineering activity. A semi-autonomous discipline, having 
both scientific and professional components, has been in the process of devel- 
opment during the past 15 to 20 years. It is too young a specialty to have 
even a reliable nomenclature for itself. It has been called variously engineering 
psychology, human engineering, human factors, human-factors engineering, 
ergonomics, and applied experimental psychology—to indicate some of the 
more popular labels. Whatever the title, however, the goal is coherent and 
consistent: it is to enhance man's capability by providing him with tools and 
tasks which are appropriate to system goals and which are commensurate with 
man's nature—his human capacities and limitations. 

Generally speaking, psychologists participate in the equipment design 
process in two major ways. First, they may function as "experts," perhaps as 
members of a team of designers, who propose the criteria on the basis of 
which practical decisions are to be made. When they are operating in this 
manner, it is apparent that they are engaged in the extrapolation of principles 
and rules about human performance which have been verified in other con- 
texts under the usual procedures of scientific inference. Their contributions 
are limited by the range and variety of knowledge obtained in such previous 
studies which can be brought to bear upon a current problem in design. The 
fact that such knowledge is limited often sets the stage for a second important 
function, which is the design and conduct of empirical studies for the purpose 
of discovering and defining new principles about man-machine relationships. 
As the problems of system design become better understood in their total 
sense, the kinds of experimental studies that the psychologist designs likewise 
become increasingly useful in their yield of generalizable information. The 
use of concepts like "information," "load," "channel capacity" in modern 
research on human performance have made possible a significant increase in 
the application of measurements taken in one task situation to those of an- 
other. Many design questions, however, still require the execution of care- 
fully controlled empirical investigations to provide answers that can become 
the basis of practical system decisions. Finally, looking forward to the time 
when the application of psychological principles can be made with maximal 


confidence, the psychologist is concerned with a 


theory of human action. 
Presumably, 


sun the degree of generalizability of findings regarding man-machine 
activities would be greatly enhanced by a comprehensive theory of human 
performance, only the barest outline of which can be said to exist today. 

The purpose of the present chapter is fourfold: (1) to define and elabo- 
rate some of the relevant psychological principles that enter directly into the 
system development activity; (2) to provide an operating framework or 
schema that indicates how these principles can be used; (3) to give some 
appreciation of how such principles are obtained and how they are validated; 


and (4) to confirm the application of such principles with some concrete 
examples of contributions to the system development effort. 
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THE CHARACTERISTICS OF THE OPERATOR 


Systems are operated by people. One must know what people are like 
if he is to design the system so that it can be used effectively. Much of the 
relevant detail of human capacities and limitations has been made explicit in 
a previous chapter (Chapter 4). However, since the point of view of the 
present chapter differs somewhat, it may be helpful to review some of the 
most critical points in this section. 


Error Analysis and Elimination. A convenient point of reference for 
discussing human operator characteristics is error analysis. In somewhat over- 
simplified terms, the psychologist's contribution in system engineering is often 
one of interposing decisions which minimize the opportunity for human error, 
while maintaining the contributory resources of the human operator. Fre- 
quently, the method chosen is merely simplification: the elimination of am- 
biguities. Since simplification is often commensurate with good engineering 
practice and economy (Chapanis, 1960), there is considerable temptation to 
limit concern to that aspect. However, as the task itself (as contrasted to the 
system intended to accomplish the task) becomes intrinsically more complex, 
it is often not possible to determine in advance what operator action will be 
“correct” or what will be “incorrect” in any given circumstance. Often the 
circumstances of activity themselves cannot be anticipated. Thus, it may be 
possible to do a very adequate job of “human engineering” on an electric 
range or a telephone handset, because the purpose, circumstances of use, and 
characteristics of the user population can be relatively well established in 
advance. In contrast, a motor vehicle perhaps exemplifies a good intermediate 
case: its operation is inherently more complex, the circumstances of its opera- 


tion are extremely varied (involving, for example, weather, traffic, roads) and 
possesses a far greater range 


the driver population, if not more varied, at least 
of goals for the use of an automobile than is the case for the use of an elec- 
tric range. 

The most complex system 
tions. The environment that inc 


s of all are those used in competitive situa- 
ludes a competitor or opponent (as in mili- 
tary operations) is likely to contain vicissitudes which are purposefully un- 
predictable. That is, the intelligent enemy will make the circumstances of 
s variable and unsettling as possible. In view of this cir- 
cumstance, it is not surprising that the man-machine systems which seem to 
attract most attention from behavioral scientists are predominantly of the 
military type. More to the point, however, is the implication that simplifica- 
tion, as such, is not an adequate answer to the systems engineering problem. 
One must think beyond mere "idiot-proofing" in order to do a reasonable job 
in developing man-machine systems which can cope with involved and indeter- 
minate situations. Such a requirement takes us back to the matter of operator 
characteristics and their effects in the production of error. 


system operation à 
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Operator Functioning in Systems. In examining the operator for the 
purpose of identifying those of his characteristics which are germane to Sys- 
tem performance effectiveness, the most productive framework appears to be 
one which involves the specification of the operator’s role or function in the 
system. While this function varies in detail from system to system, there 
appear to be some general, normative aspects. By and large, the human being 
has two interrelated functions in almost any system: he may perform an 
information-processing or “input-interpretive” role, and he may perform a 
decision-making role, in which input integration, synthesis, prediction, com- 
parison, and response selection may comprise the major subelements of action. 
The emphasis, then, in classical psychological terms, should quite obviously 
be on perceptual and cognitive mechanisms. Moreover, these mechanisms 
should be conceived as Operating in an interactive context; that is, the opera- 
tor manipulates the system equipment, and in just as genuine a sense, the 
equipment manipulates the operator. 

As Chapter 2 pointed out, traditional categories of human functioning 
for the psychologist are sensing, identifying (frequently called perception), 
and interpreting (cognition), and we continue these same distinctions here. 
What we need to consider here specifically, however, is some of the particular 
emphases that system task and job design gives to these human functions 
in their various modes. What are the questions concerning human functions 
which the psychologist meets when he faces the problem of designing equip- 
ment and tasks to optimize human capabilities? We shall describe these here 
as related to input mechanisms in identifying and interpreting, to the mecha- 
nism of memory, and to response (or output) mechanisms. 


Identification Functions 


additional emphasis. First, ther 
tifications associated with vari 


Ous sensory modes, such as vision 


capacity has distinct and stable upper bounds. 


A second area of increasing relevance to system design concerns the 


phenomenon of attention. Since it is not possible here to review the literature 
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on this topic, it will suffice to point out that interest in attention is currently 
going through a revival under the heading of vigilance research. The problem, 
stated briefly, is how to insure the attention of an operator to the highly infre- 
quent signals of a monitoring task. At first, this seems to be the opposite of 
the sensory-capacity problem: the distinction between too much and too little 
sensory input. However, there is reason to believe that altogether different 
psychological processes are involved in the two types of task. For vigilance, 
one of the most promising explanations appears to be a hypothetical inhibition- 
arousal mechanism, based on the notion that any reiterative repetition of 
either a stimulus or response raises the detection and elicitation thresholds 
respectively for all stimuli and all responses (that is, induces a state of som- 
nolence or fatigue). The organism (operator) is either aroused “all over” or, 
in a sense, encapsulated “all over.” This effect appears to depend on the 
heterogeneity of the stimulus-response pattern; the more heterogeneity, the 
greater the arousal. 

The third question relevant to the identifying function concerns percep- 
tual interference: the detection of signals in noise. Noise is here used in its 
broadest sense, as a background stimulus which can mask a critical signal, 
whether the signal be visual, auditory, tactual or some other. The problem 
of masking of signals by noise has often been studied as a problem in sensing, 
in which the basic determining factor is the similarity (psychophysical) be- 
tween the signal and the noise stimuli. However, when the task is one of 
detection (in the sense of identifying a particular signal), humans have a 
remarkable capacity for selectivity. Somehow, they are able to do a remark- 
able job of preventing extrancous material from becoming an interference to 
the detection task. Often very minute and fractionated cues presented against 
a heavy background of noise are all that is necessary to enable a person to 
reconstruct a complex picture. For equipment design, however, the question 
exists as to how to enhance a man’s ability to abstract the signal from the 
noise components of his stimulus environment; or, secondarily, how to mini- 


mize the noise itself. 


Interpreting 

available to the operator through his sen- 
sory channels, what do we know with respect to their meaning to him? Most 
of the signals in man-machine system operation are symbolic; that is, they 
have assigned rather than intrinsic meaning. The problem is Whether the 
intended meaning and the understood meaning are the same. A discussion of 
this question by Campbell (1958) emphasizes the point that the human 
operator has certain tendencies to distort the meaning of stimuli, or to assign 
meaning that was not intended. A part of this distortion originates in the 
process of identification; that is, the tendencies for redintegration, closure, 
and the like in response to fragmentary or noisy inputs constitute sources of 
perceptual errors, as suggested by the Gestalt psychologists. 


Given the fact that signals are 
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Closely related, yet generally conceived as a cognitive process, is the 
mechanism of expectation or cognitive set. People recognize what they expect 
to recognize. Such expectations can be very general and sustained; they can 
be established over brief experimental exposure (for example, the Einstellung 
effect, Luchins, 1942); or they can be imparted symbolically by instructions. 
Despite man's remarkable capabilities for adaptation or adjustment, there are 
contrary mechanisms which give both the advantages of continuity and auto- 
maticity to certain complex responses (Bahrick and Shelly, 1958) and the 
disadvantages of rigidity. 

There are emotional components involved in the interpretation process 
also. While "perceptual defense" may be a slightly disreputable concept nowa- 
days (Freeman, 1955), it still appears that our cognitive experiences are 
influenced by our motives to some extent (Eriksen and Brown, 1956). Thus 
the individual recognizes, in a meaningful sense, not only what he expects 
to recognize, but also what he wants or needs to recognize. 


Memory 


Probably the most significant aspect of memory for system design con- 
siderations, as opposed to system training (compare Chapter 2), is the short- 
term, or buffer, memory. One can consider this aspect of behavior as pos- 
sessing a capacity similar to the channel capacity of perception, although we 
know somewhat more about the buffer storage mechanism. Perhaps the most 
important factor affecting its operation is interference. It seems certain that 
a simple decay hypothesis is inadequate to account for storage loss in short- 
term memory; that is, nonretention of intelligible and significant inputs 
appears due to their displacement in storage by later arriving inputs (Bahrick 
et al., 1961). The process transpires very rapidly, and small amounts of in- 
formation are involved. A simple demonstration is provided by an attempt 

m digits. By the time one has heard the seventh or 
, the second one cannot be remembered. (The first 


take us too far afield. ) 
capability exists with re 


ab Spect to short-term memory, but it is one for which 
realisic compensations c. 


an be made in task and equipment design. 
Response Factors 


When we look at it analyticall 
man can do so many varied things 
Man is neither particular! 
actions. This, after all, 
first place—to augment t 

Two aspects of m 


y, it is always somewhat surprising that 
with such a limited effector apparatus. 
arly strong nor particularly precise in his muscular 
is the raison d'étre for man-machine systems in the 
he muscles, as well as the brain, of man. 

an’s response attributes are worthy of mention. First, 
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man's most significant response domain is language. It is in this response area 
that man's adaptability, flexibility and problem-solution synthesis capabilities 
are most apparent. In language, human beings have the widest possible range 
of response alternatives. However, the human operator can introduce his 
action choices back into the mechanical portion of the system only by overt 
manipulation of some control device. This fact differentiates the man-machine 
system from many other classes of environments. As things stand at present, 
the verbal response mode, while the most ubiquitous and elaborate output of 
the human organism, must be translated into other forms of response, such as 
turning a dial or flicking a switch, before it is accessible to the machine. This 
is not to say that the actions of the human operator cannot be expressed in 
verbal form, or that in multiman systems indirect control by verbal means is 
not possible. The point to be emphasized is that ultimately the verbal response 
must be transformed into physical motion, in order to affect machine elements 
in the system. 

As stated previously in connection with input capacity, the human opera- 
tor is conceived as a mechanism with a wide but bounded range of responses. 
As a system element, the human operator may be required to utilize only a 
small fraction of this response repertory. The speed and accuracy of a pre- 
scribed set of such actions may be the critical parameters for human respond- 
ing. In many systems, the human operator is called upon to choose from a 
finite set of possible actions; and in some systems, the operator may be called 
upon to develop his own novel responses. 

Even in the highly prescribed situation where a minimum number of 
c alternatives is relevant, however, the sequence of response alterna- 
ical to successful performance. Thus one can usefully 
regard human performance as a series of discrete actions. Skilled performance, 
particularly as exhibited in such manual skills as steering an automobile, tends 
to mask the discrete nature of the responses and their trial-and-error charac- 
teristics. The apparent smoothness and rapidity of the behavior suggests both 
continuity and stereotopy of response. The performance of even a highly 
skilled driver under stressful circumstances, as when he is driving on slippery 
pavement, however, re-exposes the basic elements of the process. Under such 
conditions the driver's movements become dislocated in sequence as they are 
interrupted by needed revisions of the normal series. Thus even seemingly 
highly coordinated, continuous behaviors may be seen as a tightly linked set 


of discrete response cvents. 


respons 
tion is likely to be crit 


i HUMAN FUNCTIONING IN SYSTEM TASKS 


The broad categories of human participation in the system have already 
been specified: they are predominantly the information-processing and deci- 
sion-making functions. In a word, in spite of the advances in computer tech- 
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nology, man is the “brains” of any system. His role may well become 
progressively more remote from the moment-to-moment operations as com- 
puters are developed to take over more of the subordinate functions. But the 
executive role, especially with respect to the specification of goals, will pre- 
sumably always be man's prerogative. Computers which can learn, in the 
sense of being able to adopt new means (a potential but as yet unrealized 
characteristic), are not expected to be capable of setting their own goals 
(compare Chapter 3). 

For the present, we may safely concern ourselves with attempting a 
generalized description of some of the most important varieties of information- 
processing and decision-making functions that are typically allocated as tasks 
to human operators within Systems. The categorization used here is an arbi- 


trary one, and it is not intended to describe complete tasks assigned to human 
operators. 


Information-Processing 


Signal Detection and Classification. Except in the carefully contrived 
environment of Sensory deprivation studies (see Solomon et al., 1957), an 
individual is under continuous bombardment of stimuli in all the sensory 
modes. He must separate from this "blooming, buzzing confusion"? that which 
is relevant. He must isolate the "signal" from the “noise.” Once the presence 
of a signal is established, the signal must be. classified or given meaning. 
Signal classification is a typical identification task, in which differential re- 
sponding to different signals is the performance expected. Thus detection is a 
matter of distinguishing signal versus noise, while classification is a matter of 
distinguishing among signals. 

The sonar operator's task in an antisubmarine warfare System is a good 
example. The sonar operator looks at a moving oscillograph (paper tape) 


play output: these are targets, which the operator must di 
background clutter. Then there are subtle differences 
nals. He can make appropriate responses to indicate 
surface vessel, a Conventional submarine, or an 
times he can make even finer distinctions. Syst 


among such target sig- 
that a given target is a 
atomic submarine, and some- 
em design can either help or 
ection and classification func- 


tions, particularly by enhancing the distinguishability of the wave forms dis- 


played on the tape. 


Recoding. One of the conceptually r; 
information processing is that of translation, 
who receives in Morse code provides one insta 


ather simple requirements in 
or recoding. The telegrapher 
nce of this kind of task, which 
requires him simply to recast the dots and dashes into English words, He need 
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not be concerned with the message content, and indeed may not remember 
what he has translated. Nevertheless, it is apparent that this task is aided by 
the use of the interpretation function, which provides word meanings and 
probable word sequences. Codes vary in the degree to which they can be 
conveniently interpreted: some are quite meaningless in the sense that they 
are completely arbitrary. Some codes, on the contrary, are constructed on the 
basis of learned associations already available to the operator. An example of 
this latter situation is provided by the recent development of a graphic code 
for use in cathode ray tube displays; symbol forms were selected on the basis 
of their compatability with existing response tendencies (Howell and Fuchs, 
1961). The format of the codes and the degree of the correlation between a 
signal in one code and a signal in the other are design factors that have 
obvious implications for performance effectiveness. Many of the findings 
from the extensive literature on associative learning are relevant to such 


design considerations. 


Accumulating and Summarizing. In some traditions, such as computer 
technology, transformations of information by accumulation and mathematical 
manipulations are the essence of information processing. Indeed, such arith- 
metic and statistical tasks are increasingly allocated to machine rather than to 
human components. However, it is well to understand the processes in- 
volved, since some traces of tasks of this type may remain man's prerogative 
in even highly automated systems. Morcover, in some systems, such as large- 
scale business and military organizations, the raw inputs are nonquantitative 
or only partly quantitative, and therefore may: not be susceptible to machine 
manipulation. i T 

An example of the latter occurrence derives from an analysis of military 
intelligence systems. Part of the total intelligence picture of a foreign nation 
would be the attitudes of a populace toward their current regime. Barring 
elaborate attitude scales, such incoming intelligence data would be highly 
qualitative. Nevertheless, some summarization would be necessary to make 


the information ultimately usable. Human operators would need to go through 


a quasi-statistical operation, from which summary information such as “aver- 


age” attitudes would emerge. A m ah " 
The general process is further exemplified by the tradition of "briefing. 


Higher-management levels in an organization or system are kept abreast of 
the status of subordinate units by presentations of selected items or sum- 
maries of activities. The format may have a distinctly statistical flavor by 
including such things as average productivity rate and accident frequencies. 

The system design problems associated with this type of task are pre- 
dominantly those of allocation. To what extent can and should such opera- 
tions be automatized? Secondary issues involve the design of sensing and 
measuring equipment and displays and the establishment of optimal reporting 


Procedures. 
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Output-Processing. While the examples and terminology employed up to 
now have emphasized the input side of information-processing in the man- 
machine system, it should also be remarked that the system itself generates 
information as well as accepting information from its environment. Some of 
the signals generated within the system may be conceived of as system- 
regulatory in character. Such signals are for “internal consumption only” and 
contribute to the maintenance chores that are a part of every system opera- 
tion. Other such system-initiated messages make up the output of a particular 
subsystem, and such material may Tequire processing before emerging from 
the system as a whole. Classifying, encoding, and accumulating are frequently 
required. In a ballistic missile launching system, for example, the “output” 
portion of system operation often appears to contain most of the stress and 
drama. Actually, since the weapon is not manned and is aimed at a preselected 
target, the “count-down” is the most dynamic man-machine system contribu- 
tion. The count-down is essentially an output activity based on inputs gathered 
over an extended time period. Thus, information processing can in reality be 


one of the most highly essential components of what appears to be an output 
operation. 


Value Weighting and Destination Routing. The information processor 
and the decision maker may be, in some man-machine systems, one and the 
same individual. When they are not, there is usually a transitional process 
whereby the decision maker, in effect, continues the information-processing 
function for his own particular ends. Underscoring the continuity between 
information processing and decision making, there is at least one kind of Sys- 
tem task that clearly partakes of both. Once the information is “in the sys- 
tem," it must be acted upon. Although decisions are involved, they pertain to 
the message, rather than to the environment the message purports to describe. 

While closely related to the signal classification task, the activities of 
weighting and routing deserve particular attention. The two 
weighting and routing may be called "scheduling" tasks, The operator deter- 
mines the importance and destination 
ranges the sequence of messages in the ch 
characteristics that can influence the effi 
least of which are the cl 
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Selection and synthesis. In performing decision-making tasks, the indi- 
vidual may not treat equally the total output of whatever source of informa- 
tion he has. He may select and reject various of his incoming messages, and 
he will certainly add his own evaluation to that which has already been done. 
His criteria may be different and superior to those of the subordinate infor- 
mation processor, since he is likely to have overlapping information from a 
greater variety of sources. He can thus interpret the content, and include his 
experience with respect to the consistency and veracity of alternative sources 
in his assessment. For example, the battlefield commander may well have 
several subordinate agencies feeding in information with overlapping content. 
The perspective of these agencies will vary; air reconnaissance will reveal 
some aspects of the battlefield situation, and ground patrols will give other 
aspects. Neither perspective alone could yield a "correct" picture. Both may 
contain incongruities or fragments that fail to make sense in a historical per- 
spective. The commander or his staff must select out the useful from the non- 
useful and put the useful pieces from various sources together. 


Pattern Construction. Closely related to selection and synthesis is the 
decision-making task of creating a coherent whole out of discrete fragments. 
In any very complex system, a picture of the state or condition of the system 
environment is built up like a complex jigsaw puzzle in which competing 
pieces fit the same gap, all the pieces are not available, and the picture itself 
is changing from moment to moment. In the battlefield example just cited, 
the data received from even a very comprehensive intelligence and reconnais- 
sance effort would be incomplete. The completed image of *what is going on 
out there" is, then, partly logical, rather than empirical; the gaps are filled by 
sense," rather than solely by what is known. In classical psy- 


what “makes N ; 
action partakes of redintegration on a grand scale. 


chological terms, this 


Cause-and-Effect Attribution. Once a coherent picture of conditions 
exist, the human decision maker is forced to assume or determine, as best he 
can, why conditions are as he perceives them; that is to say, he must under- 
take to ‘classify inputs in terms of their effects, as opposed to their appear- 
ances. Let us consider an example. A military command-control system is 
he tactical air support of an infantry operation. Some of the 
environmental factors of concern would be disposition of enemy troops, hit 
frequency on assigned targets, proportion of secondary targets attacked, pro- 
portion hit, and cost of operations. As each bit of data is presented to the 
commanding officer, he must, as a matter of course, ask and derive a hypo- 
thetical answer to the question of why. Only through dealing with or acting 
upon the cause-and-effect dynamics of its environment can a system function 


effectively. 


responsible for t 
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Time-Line Analysis and Prediction. Coordinate with the attribution of 
cause to observed events is the evaluation of time-contingent processes in the 
environment of the system. Many phenomena for which either the immediate 
or ultimate causes cannot be easily determined are time-contingent. Organ- 
ismic growth or the fluctuations of the stock market are examples. The reason 
why the changes take place are not known, but it is often possible to predict 
what they will be, since they have some orderliness against a time base. The 
decision maker in a man-machine system has both the ability and the neces- 
sity to consider time-linked phenomena, and to extrapolate from observable 
trends in completing his picture of the system's operation. 


Critical-Cause Selection. As the complex picture of conditions ap- 
proaches completion, there is usually a necessity to compare that picture with 
that which is desired or required by the avowed goals of the system. In per- 
forming in this way, the human being must assess the discrepancy, if any, 
and focus his attention on the dimensions of that discrepancy, whatever they 
may be. It is, obviously, the cause of the particular discrepancy with which 
he must contend. Moreover, he must at this stage begin to consider his re- 
sources with respect to what needs to be done. More often than not, his 
Tesources are not sufficient to do all that is immediately necessary. He must 
then determine a priority system and decide what is critical for immediate 
action. In essence, he must determine what factor or limited combination of 
factors in his task environment can be manipulated, given his limited re- 
Sources; he must next decide which of these potential manipulations will lead 
most directly to his goal. For example, a military unit commander may decide 
that an enemy installation atop a given hill is the factor retarding his advance. 
Such factors as his own troops' fatigue, bad weather conditions, and the like 
may be highly contributory, but they are bcyond the decision maker's control 


or not compatible with goal achievement. He must thus select a critical cause 
for the allocation of effort. 


Action Selection. It would be a poor system indeed that had but one 


response for all occasions. Most systems are typified by a multitude of means 
for each end; the particular means and the extent of its use to be determined 
by the decision maker’s assessment of conditions. Indeed, the most interesting 
systems are those having what might be called modular or “building-block” 
means, such that novel combinations or patterns of response can be put 
together to meet virtually any exigency. The performance of the human deci- 
sion maker faces its crucial test when he makes his choice of action from 
among the alternatives, few or many, 
example is provided by an airline mana 
aircraft to different routes. With a single ai 
line could not be operated very efficie 
decision maker would h 
lems as high-traffic sho 
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synthesize a solution by converting some of his aircraft to cargo or mixed 
cargo-and-passenger carriers. 


Effect Evaluation. The loop is closed by the consequential feedback 
subsequent to action initiation. The decision maker can assess, through the 
effects wrought by his action upon the system environment, whether his per- 
ception of the environment was valid and whether his choice of action was 
correct, Did the impact of his selected action have the desired effect on the 
environment? If not, was the direction of effect appropriate but the extent 
too great or small? Such questions ordinarily cannot be answered in a single 
cycle of action-reaction, of course. But the process of system operation is a 
continuous action, providing the opportunity for the decision maker to assess 
his hypotheses and response predilections and to revise them. Thus, the man- 
machine system has the potential for continuing development through action. 


THE DESIGN OF HUMAN TASKS 


fal 


Having described some categories of human tasks in systems, we can 
turn now to their interpretation for system design purposes. We shall attempt 
to fit the task categories discussed in the previous section into the mold of 
system design requirements in the realm of task and job definition. Although 
general rules usually suffer from oversimplification, some degree of simplifica- 
tion nevertheless appears appropriate in this discussion. To this end, we shall 
consider only the major functional categories of information processing and 
decision making. From what is known concerning the activities required of 
human participants in man-machine system operations, as well as of human 
capabilities and limitations (see Chapter 4), we are now ready to describe 
the principles that enter into the process of designing human tasks. 


Design Factors in Information-Processing Tasks 


A most important principle relevant to designing tasks involving infor- 
mation processing can be reduced to the short phrase load constancy and 
input variability. By load is meant the momentary demands placed upon the 
operator's limited identification and associated memory capacities. Load needs 
to be constant, in the statistical sense of not being susceptible to violent. fluc- 
tuations; it is also intended to imply that the level of load should be "inter- 
mediate." When the signal input rate exceeds the perceptual/mnemonic 
capacity of the operator, not only are the excess signals "lost," in the sense 
that they are not detected and identified, but they also come to function as 
additional noise. When this happens, these excess signals act as distractions, 
and they have the effect of masking residual signals that otherwise would be 
within capacity limits. 


In the opposite case, when the input signal rate is low, loss of vigilance 
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(breakdown of filtering sets) takes place. The behavioral mechanisms which 
account for these events are subjects of theoretical controversy, but the phe- 
nomenon itself is demonstrably valid, and the effect occurs across sensory 
modalities. Thus, for optimal task design, the rate of signal presentation 
should be at a steady, intermediate level. 

Methods of display of signals—such as their coding and the presence of 
noise—will be discussed in a subsequent section. However, a general rule 
concerning the nature of signals is germane to the problem of task design. 
It would appear that a good case can be made for the desirability of content 
variability of signals. Without content variability, the system is exposed to 
the possibility of "getting in a rut." There are many anecdotal instances which 
corroborate this notion. One need only visualize the automobile driver who 
uses his car only to drive from home to work and back—never anywhere else. 
Were such an individual to move his residence or place of work after many 
months of such a practice, his response-error frequency, and the delay occa- 
sioned by his equivocation, would seem likely to be higher than they would 
be had he driven a wider variety of routes under a wider variety of circum- 
stances. 

The concept of habituation also seems to be relevant to the problem of 
task design. In fact, it can scarcely be overemphasized that the system, insofar 
as its human components are concerned, is always in the process of learning, 
The conditions which are experienced by the human operator on any occasion 
Serve to determine his subsequent reactions, whether this is intended to be so 
or not. If a single or highly limited range of conditions is displayed and a 
restricted set of responses is emitted, there will probably be resistance to 
change, or to the accommodation of new conditions as they arise. In order to 
insure that a system will be capable of utilizing its full capacity and response 
repertory when it must, some variety in signal display is essential. This does 
not necessarily mean that the precise conditions of infrequent events must be 
artificially introduced into the system inputs. Rather, it means that sufficient 
variability needs to be included in order that a tendency is maintained by the 
human operator to “expect the unexpected.” 


Design Factors in Decision-Making Tasks 


For tasks involving decision-making functions, there is.a second prin- 
ciple, cast in the same form as the first. It is goal specificity and means flexi- 
bility. Goal specificity does not mean that the System must have but a single 
goal, nor does it necessarily imply that multiple goals be altogether com- 
patible. It simply means that the ultimate objectives of the system, the various 
subgoals, as well as the interrelationships among these, need to be explicitly 
communicated to the decision maker. 

The reletionship of means to ends, however, should be left as unspecified 
and flexible as possible. Of course, the responses available to a decision maker 
are always limited. They are limited by his own repertory and imagination, 
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by the facilities at his disposal, and often by administrative fiat or organiza- 
tional directive. The last-named reasons, however, are often the least 
justifiable. 

Our reasoning with regard to decision-making tasks parallels that used 
to explore the information-processing function in task design. The “what” of 
a task should be clear (and having a correlative “why” helps); but the “how” 
should be left as open as possible. The decision maker must have a standard 
or criterion if he is to function as an adaptive unit, as is implied by the cyber- 
netics or servomechanism model. If the means are designed to make optional 
behavior possible, the consequent flexibility makes provision for such factors 
as individual differences among operators and unpredictable combinations or 
patterns of operating conditions. 

It will be useful here to describe one example. Experimental studies of 
human activities in radar air-traffic control have used the general variable of 
“procedural flexibility” repeatedly. One of the earlier ones in a series devoted 
to this operation explored the effect of giving operators a set of fixed flight 
paths to use in guiding aircraft in a blind landing situation, as opposed to a 
condition in which no fixed paths were available. The hypothesized trade-off 
centered on the possibility that fixed paths would reduce confusion and help 
the controller keep track of a heavy traffic load. He was expected to use paths 
and check points to organize the traffic flow. It was discovered, however, that 
the operator's performance—as reflected by flight delays and by frequency 
of near misses (midair conflicts )—was relatively poor under these rigidly set 
conditions (Kinkade and Kidd, 1959a). In the terminology of the operator, 
he “had no elbow room” to take advantage of exploitable situations. Even in 
the absence of continuous target identification, the highly structured configura- 
tion of an approach path did not help. The means provided were too inflexibly 
specified. 

The main import of these findings was corroborated in a later experiment 
(Howell, Christy, and Kinkade, 1959). Under normal conditions of operation, 
maximum freedom of action was facilitative to performance. Under emergency 
conditions (radar failure), increased specificity of the means of action was 
useful in providing a substitute for adequate informational inputs. It was in 
the area of communications, rather than in the decision-making domain, that 
procedural rigidity resulted in greater effectiveness, even under conditions 
which were intended to capitalize on the structured aspects of the situation. 
Other studies in the series provide further substantiation of the main point 


(Kinkade and Kidd, 1959b; Kidd, 1961). 


DESIGNING EQUIPMENT FOR HUMAN USE 


Having stated some general principles applicable to the design of human 
tasks, we can turn our attention to the ways in which design is actually made 
concrete within the system development process. It is apparent that the human 
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operator, whether functioning as an information processor, a decision maker, 
or both, occupies a position as a link between two other portions of the 
system. This means that he (1) responds to the preceding unit's output as his 
input and (2) by his action provides an input to the next unit. When provided 
by a machine, the configuration of output events that constitute input to the 
human operator is generally called a display. The physical objects which he 
operates (particularly with his hands and feet) in order to provide an input 
to the next unit in the chain are called controls. Obviously, the way the human 
operator must function within the system will be determined by the nature of 
these displays and controls. Accordingly, considerations of effective design 
for the man-machine configuration usually result in decisions concerning the 
physical characteristics of these aspects of equipment. 


Display Design 


The goal of display design is to provide the operator with usable informa- 
tion germane to his task within the system. One can usually begin with the 
assumption that the system has at the outset the basic means of acquiring all 
the information that might conceivably be useful. But once the information 
is attained, how and in what manner should it be distributed among, and pre- 
sented to, the human elements of the system? The problems can be defined 
somewhat more specifically. The variables of interest in the design of displays 
have been classified in the following ways (Williams, Adelson, and Ritchie, 
1956): 

1. Readability, legibility. Obviously the operator must be able to hear 
or see or in some other way sense the signals being provided for his use. He 
must also be able to sense differences among different signals; variables con- 
cerning both the display proper and the viewing or sensing environment (for 
example, illumination) come within this class. 

2. Sensory modality. The question raised by this category concerns 
which sensory mode should be employed to convey various kinds of infor- 
mation. 

3. Multiparametric or combined displays. Here are included questions 
of what and how many different kinds of information can be incorporated 
within a single display and how this is to be accomplished most effectively. 

4. Display coding. This category implies questions as to the language 
form or the kind of symbols to be employed in presenting information. 

5. Filtering. Questions of this class concern ways of preselecting infor- 
mation inputs so as to simplify the interpretation task. 

6. Clutter and noise. Included in this category are problems pertaining 
to the elimination of false or masking signals in the display. 

Let us look now at a specific case or two, in order to gain an idea of 
the steps involved in dealing with such problems. The caution should he 
given, however, that the separation of display design, control design. and 
procedures specification, while useful for purposes of analysis, is actually 
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somewhat artificial. System design, as opposed to component design, stresses 
the total system and the total compatibility between all system elements. 
Though these elements may at times be studied in isolation, one should not 
lose sight of the reintegration that must eventually be accomplished. 


Instruments. For several reasons, the most intensively investigated topic 
within the class of display-design problems has been that of airplane cockpit 
instruments. During the latter stages of World War II and subsequently, as 
aircraft continued to become faster, heavier, and more complex, instrument 
reading errors became less and less tolerable. Since pilots were being trained 
in large numbers and often being placed in stressful situations with minimal 
experience, dramatic instances of instrument-reading errors became almost 
commonplace. The responsible military agencies turned to psychologists to 
help with solutions to these problems, with the effect that many of the pio- 
neers in engineering psychology were introduced to their eventual specialty 
through this particular kind of work. A classic example of cockpit instrument 
evaluation is Grether's (1947) study of altimeters. In that experiment, nine 
alternative altimeter configurations were compared with respect to both speed 
ccuracy of reading. Both experienced pilots and inexperienced students 
were used as subjects. The standard three-pointer circular dial altimeter 
(which is still widely used) was found to be among the poorest of the dis- 
play alternatives. A counter, moving tape, or a combination of counter and 
single-pointer dial were among the best alternatives. The difficulty with the 
multiple-pointed instrument can probably be accounted for, at least in part, 
in terms of its demands on the short-term retentive capacity of the operator. 
With such an instrument, the operator must read one pointer, store that 
information, read the next pointer, store that information, and finally read 
the last pointer. Having done this much, he has then assembled the necessary 
digits of the altitude figure. In addition to this basic difficulty, the operator 
must not confuse the order of the pointers (the shortest pointer reads in 
10,000’ increments, the medium sized pointer in 1000’ increments, and so 
on) and must be able to interpolate when a pointer rests between two scale 
ation of dial numerals can also contribute to the 


and a 


graduations. Pointer obfusc 


difficulty of reading this instrument. 

Obfuscation of scale marks and the need for interpolation are related 
by an interesting example of a “trade-off” relationship. If scale units are too 
close together, obfuscation results; if they are too far apart, readings based 
on interpolation decline in accuracy. Thus, a graphic plot of reading errors 
against the size of the space between graduations yields a U-shaped function 
with a minimum between .05 and .07 inch (Grether and Williams, 1947). 
Related research on dial reading has also been carried out by Kappauf and 
Smith (1951) and by Kappauf, Smith, and Bray (1947). 

Another interesting aircraft instrument problem centers on the attitude 
bank and roll indicator. Loucks (1947) reported a carefully executed series 
of comparative evaluations of various attitude indicators. He used a group of 
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137 air cadets with an average of 6.5 hours of flight training. A Link Trainer 
was used to provide the task situation. Loucks found that an arrangement in 
which the horizon line is fixed and the aircraft symbol moves in the same 
direction as the roll of the aircraft was reliably superior to the standard 
instrument, in which the aircraft symbol is fixed and the horizon line is the 
dynamic element. Measures of the maintenance of level flight by the operator 
in standard trainer missions were the criteria of performance employed. 

Despite this carefully tested finding, the design implications of this study 
remain somewhat equivocal for two reasons. First, the subject population was 
composed entirely of novice aviators. Transfer of training on the part of expe- 
rienced, senior pilot personnel, who were accustomed to the standard instru- 
ment, was not measured, but might be expected to be of the negative variety. 
Second, the task situations as presented by the Link Trainer was greatly sim- 
plified relative to actual instrument flying. Although such abstraction often 
characterizes a sound research approach, the method did not provide a way 
to assess possible interactions between the varied and complex circumstances 
of instrument flight and the particular design in question. The study never- 
theless remains a classic in its demonstration of a method for approaching 
such display problems and finding meaningful answers by the use of measures 
of human performance. 

A recent proposal (Fogel, 1959) has been made to combine the fea- 
tures of both instruments. The so-called Kinalog display system shows the 
aircraft symbol as moving during the first few seconds of a coordinated turn. 
As the maneuver Progresses, the horizon line begins to tip from the hori- 
zontal until, at the completion of the maneuver, the aircraft symbol has 
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Radar Displays. Let us turn to 


l a second major topic in the display design 
area: the design of radar and other 


ground-based system displays. Coming as 
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it did in the early years of World War II, radar provided a novel, fresh, and 
therefore desirable testing ground for many advancing technologies, including 
that of applied psychology. In several series of experimental investigations, 
both the display itself and the viewing environment have been examined, in 
an attempt to make such devices compatible with human characteristics and 
task requirements. One relatively recent example of such studies will be de- 
scribed here as an illustration of method employed. 

The problem centered on the presentation of altitude information to 
radar air-traffic control operators. Ordinarily such operators are given the 
lateral position of the aircraft under radar surveillance through a cathode ray 
tube display. Since the display face is a flat surface, it provides no direct 
altitude data. In order to get altitude confirmation, the operator (controller) 
must query the pilots of each aircraft by voice radio. Voice query and reply 
are rather time-consuming and generally require five to six seconds at a mini- 
mum. Accordingly, it seemed reasonable to propose that a visual display 
which provided continuous altitude information would enhance the effective- 
ness of the system. 

Such a display was designed and constructed, and then tested in the 
laboratory in a detailed simulation of the radar approach control operation 
(Schipper er al., 1957). It was found empirically that little, if any, perform- 
ance enhancement occurred when the visual display containing altitude infor- 
mation was employed. Since the PPI display and the altitude display were 
contiguous but separate instruments, it seemed likely that the visual time- 
sharing load imposed by this arrangement offset the time gained by direct 
access to the altitude data. Consequently, a series of laboratory experiments 
was conducted to determine how the display layout could be modified to com- 
bine lateral position and altitude information in a single display. Several such 
formats were evaluated under a number of explicit task requirements. It was 
observed that a very substantial interaction obtained between the display 
design characteristics and the various task modes, insofar as speed and accu- 
racy of operator performance were concerned (Kraft, Boyes, and Alluisi, 
1959). ; e 
It may be noted that these experiments did not yield a single, unequivocal 
answer to the general question of the absolute utility of the visual display of 
altitude information. One of their main contributions, aside from the specifics 
of the test setting, is in suggesting a research approach. But beyond this, their 
findings reaffirm the caution against arbitrary isolation of system components 
and task elements in designing displays for human use. In all cases, the results 
showed optimal design to be dependent on an interaction between equip- 


ment, task requirements, and operator characteristics. 
> 


Computer Displays. A third example of display design derives from 
the context of the increasing tendency to employ computer display combina- 
tions in complex decision-making tasks. In order to understand the problem 
faced by the system designer of such a combination, one may consider the 
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job of a corporation vice-president in charge of sales, with his data inputs 
automatized and the pace of his job accordingly multiplied a hundredfold. 
A man in this position is faced with the problem of making a number of 
decisions on the basis of a large amount of information. Some of this infor- 
mation is continuous, some is discrete; some of it is determinate, some is 
indeterminate or probabalistic at best. For some of the data presented, the 
evidence is complete, for some the evidence is spotty or sporadic; for some 
the validity of the data and the veracity of the source is known, and for some 
the validity is unknown and the sources are of dubious veracity. Imposing 
automatic data-processing and mechanized display facilities on such an opera- 
tion is obviously fraught with difficulty —yet similar operations are being auto- 
matized in military operations. 

The processing and display elements of this kind of system are obviously 
intended to simplify the human's decision-making task. The simplification 
takes place in the form of both logical and mathematical operations performed 
by a computer. For example, if our vice-president were faced with assigning 
a set of new salesmen to various territories, factors such as the differences 
between the salesmen in ability and personality, the past relations of the firm 
in the market areas, and the sales potential of each area would have to con- 
tribute to his decision. A computer might act on these factors and generate a 
set of “acceptable” alternatives much fewer in number than the original set of 
combinations of salesman and territory. The vice-president could then employ 
yet a different set of criteria (for example, how the salesmen’s respective wives 
feel about their husbands’ being out of town for long periods) in making the 
final choice. 

Somehow, the computer display combination has eliminated much of the 
ambiguity of the original situation. The computer has already made a large 
number of “formal decisions” in its rejection of the largest portion of the 
possible alternatives. A question remains, however: is such predigestion of 
the information-processing and decision-making task compatible with the 
nature of the problem and the characteristics of the final arbiter, the human 
decision maker? 

Since our problem was hypothetical to begin with, we can dispense with 
it in summary fashion. Insofar as the data inputs to the system meet certain 
criteria of scalability and quantifiability, and insofar as there is a known pat- 
tern of cause and effect relationships, computerization of the problem is pos- 
sible and, in many instances, advisable. The advantage of using computers, at 
the present stage of their development, resides in their capacity to do reitera- 
tive work rapidly, without error and without complaint. Otherwise, computers 
are governed by the logic of the program and will do only what the program 
can do (see Chapter 3). When the vital question is asked, “Can the operation 
be programmed?” what is meant is: “Can the process be accurately repro- 
duced, using a fixed set of rules, logic, and computational operations?” An 
increasingly frequent possibility of error in system design appears to stem 
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from the fact that human functions which cannot conveniently be programmed 
may either be simply ignored or grossly distorted by the designer, in order to 
make them fit the criteria for computerization. 

Human characteristics relevant to the task provided by a computer dis- 
play raise other issues. For example, under normal operating conditions, how 
well does the executive conceptualize and interpret the cause-and-effect rela- 
tionships of his task that are susceptible to formal logic and computational 
treatment? Can he distinguish between the logical and “intuitional” aspects 
of his task? What kind of subroutines does he use when faced with such 
situations? Assuming that he has experienced similar problems in the past and 
has developed a consistent approach, is it possible that computerization will 
distort the decision-making context to the extent that the existing approach 
is invalidated? If so, can the executive be trained, using his past experience 
as a basis, to make eflective use of the predigested materials he gets from the 
computer display facility? At present, it can only be said that none of these 
questions has been directly subjected to psychological experimentation, al- 
though they appear to be eminently susceptible to that form of evaluation. 
Equipment design must currently proceed by extrapolation from research 
findings obtained with different ends in view. 


Design of Controls 


We now turn to the effector end of systems operations and consider the 
controls by which the decisions for action are actually carried out. Again, we 
avoid detailed descriptions of human control movements, such as are con- 
tained in handbooks (for example, Woodson, 1954), in order to achieve 
general orientation and widest applicability. 

The ruling concepts in control design should be order, coherence, and 
Organization. Rather than flexibility or changeability, one wants control de- 
vices to have the properties of being orderly and consistent in their operation 
and action consequences. From the standpoint of the human operator, per- 
haps predictability may be thought of as the most desirable characteristic. 

The layout of control panels and consoles is a good place to begin con- 
sideration of control design problems. The basic technique for the designer 
consists in analyzing the task. The task analysis provides a map of what the 
Operator is supposed to do in carrying out his job. Traditionally, the analysis 
is a description of isolated actions in sequence. For relatively simple tasks 
al assembly) the classic therblig of industrial engi- 
neering is appropriate (Gilbreth, 1911). With increasing operational com- 
plexity, as well as the necessity to develop equipment for tasks which are 
almost entirely novel, newer techniques are needed (see Chapter 6). By what- 
ever means obtained, however, an analytic map of the task is essential. 

Just as the system as a whole may be functionally organized according 
to operations, the control layout can be so organized. Functional grouping, 


(for example, mechanic 
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ease of access, differentiability of suboperations, and the like then come into 
play as criteria for console design. Unfortunately, the criteria are not always 
(in fact, are rarely) independent of each other. Functional grouping and 
frequency of use may dictate contrary arrangements. 


Missile Guidance. In a hypothetical case of a guided-missile system, an 
operator might have three main suboperations in the control guidance of a 
single weapon. These could be prelaunch checkout, flight control, and arming 
control. Let us say that the operator spends six tenths of his time in prelaunch 
checkout, three tenths of his time in flight control, and one tenth of his time 
on adjusting the warhead. Each of these main operations might be equally 
complex in terms of the total number of discrete suboperations involved. It 
is conceivable, however, that one of the arming control suboperations (for 
example, proximity-fuse adjustment) might take up almost all the total frac- 
tion of time devoted to that main operation. Proximity-fuse adjustment might 
even be the one single most frequent action in the entire task. How, then, 
should the control panel be arranged? Should the main suboperation of pre- 
launch checkout be given a priority position because all its suboperations add 
up to the biggest block of time? Or should arming control be given precedence 
because one of its suboperations is the most frequent of all? Then again, the 
flight control operation might require the most delicate and exacting control 
settings and thus be a suitable candidate for maximum accessibility. Obvi- 
ously, with so many factors to consider, any given system may require some 
very insightful analysis or, in many instances, specific empirical comparisons 
of alternative configurations. When the rules of functional arrangement are 
freely applicable without internal contradiction, they have every appearance 
of consisting entirely of simple common sense. Indeed, such is the case. The 
specialist in human design factors makes his contribution, whether it be by 
means of an analytic or an empirical study, when the straightforward applica- 
tion of rules cannot be accomplished. 


Principles of Control Design. Certain principles of control design have 
by now attained the status of familiar and frequently employed rules of thumb. 
Although some of these rules have been implicit in the preceding discussion, 
they need to be stated here in more complete form: 

1. Accessibility. All control should be accessible to the operator with- 
out requiring that he contort himself or otherwise take an awkward stance 
or position. 

2. Functional arrangement. This principle contains several parts, as 
follows: 


a. Controls should be grouped according to logical and easily com- 
prehensible categories. 

b. The categories should be based on the system or subsystem goal 
structure (that is, on the operations such as fueling, guidance, 


—J. S. Kidd - 181 


arming); not on the action called for (such as switch setting, 
vernier adjustment, rate control). 

c. Priority in location should be given to controls which require the 
most frequent use. 

d. Priority in location should be given to those controls which require 
the greatest deftness of action or which are the most difficult to 
handle. 

e. Critical controls—those having a “life-death” effect on the success 
of the operation—should be prominently placed. 


3. Differentiability. Controls should be arranged so that their differen- 
tiability is enhanced by their location. The purpose of the control should be 
apparent by its location, physical characteristics, color, mode of use, and the 
like. The coding and cue-enhancement logic should be consistent for a given 
station. (Ideally, coding would be made consistent across systems to facilitate 
operator training and transfer of skills.) 

4. Safety. Provision should be made to prevent the accidental (non- 
intentional) activation of any control. Critical controls should be covered or 
protected under lids, and provision should be made for locking some types of 
controls. 

5. Reliability. Controls should, when possible, signal their state. As an 
example, the “on-off” positions for a switch should be clearly marked. The 
mode of action correlated with direction of turn should be clearly indicated 
on all knob controls. Mnemonic cues should be included when feasible, in 
such a way that desired-setting or last-setting data are available in proximity 
to the control device. 

6. Display compatibility. In the total configuration of displays and con- 
trols making up the operator station, display control arrangement should be 
correlative; that is, controls governing the process being displayed should be 
in proximity to their related display. Display format and content should be 
dimensionally similar to control location and direction of action. 

As our previous discussion has suggested, it is often not possible to fol- 
low all these rules with equal vigor. The design of controls and control layouts 
is, of course, a matter of practical compromise, as is true of other aspects of 
equipment design. Nevertheless, this set of principles represents the factors 
that are based on empirical findings of studies of human functioning, which 
Can successfully be brought to bear on design decision having the aim of 


Optimal system effectiveness. 


REDUCING HUMAN ERROR 


[a] 


On the whole, our discussion of design problems has been carried out 
in the context of human functioning, and particularly in consideration of the 
Ways of eliminating the kinds of functioning that result in error. The analysis 
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of possibilities of operator error leads to the conclusion that its causes may 
often be identified as deficiencies in equipment design, whether of displays, 
controls, or the expected interactions between these two types of elements. 
The information obtained from such analyses is exhibited in summary form in 
Table 5.1. It will be noted that many of these difficulties provide the possi- 
bility of correction by means of equipment design (for example, unclear code 
form), whereas others would appear to be avoidable by the provision of 
external instructions, possibly by means of job aids (for example, inappro- 
priate filtering set). In still other instances, it appears that corrective action 
would take the form of training to be undertaken after the equipment con- 
figuration has been determined (for example, action-control relationships not 
understood by the operator). 


Table 5.1 


Typical Human Errors in Equipment Operation and 
Their Possible Causal Factors 


Type of Error Possible Causal Factors 


Failure to detect signal Input overload 
a. Too many significant signals 
b. Too many separate input channels 
Input underload 
a. Too little variety of signals 
b. Too few signals 
Adverse noise conditions 
a. Poor contrast 
b. High intensity of distraction stimuli 


Incorrect identification Code form or typology unclear 
of signal Lack of differential cues 
Inappropriate filtering set (expectation) 
Conflicting cues 
Conflicting identification requirements 


Incorrect value-weighting Nonlinear predictions required 
or priority assignment Multiple or complex value-scaling required 
Values poorly defined or understood 
Contingencies vaguely defined 


Error in action selection Matching of actual and required patterns faulty 
Consequence of courses of action not understood 
Appropriate action not available 
Correct action inhibited 
a. Cost considerations 
b. Procedural prohibitions 


Error of commission Correct tool or control not available 
Action-control relationship not understood by operator 
Action feedback unavailable or delayed 
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It seems evident from the table that there are a number of ways of pre- 
venting, minimizing, or reducing the deleterious effects of operator mistakes, 
when one recognizes that such errors may be understood as matters of inade- 
quate functioning of human information-processing and decision-making 
activities. In many cases, the avoidance of faulty human functioning can be 
specifically related to the design of equipment displays and controls, and par- 
ticularly to the extent to which they define sensible human tasks. If one looks 
beyond these “rules of thumb,” he can foresee the possibility of a systematic 
theory of human performance, for which currently acceptable categories of 
human functioning provide only the bare framework. 
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THE PROCESS OF SYSTEM DESIGN IS RE- 
flected in two sets of specifications: one for equipment, 
and another equally important set which describes 6 

human tasks. Just as the former descriptions guide the 

development of equipment components, the latter con- 

stitute the basic reference for all of the actions that 

determine the capabilities of the human components of 

the system. Task descriptions are the statements of those events which con- 
stitute the interactions of men with machines and with their system environ- 
ments, including other men. They are, of course, continually revised and 
refined as system development proceeds. They are necessary in order that 
system developers can provide themselves with answers to continually recur- 
ring questions about what functions men are performing in any part of pro- 
jected system operations. In addition, they form the point of origin of most, 
if not all, the procedures used in developing the “man” part of the system, 
the personnel subsystem. 

Once it is known what men will do in the system, it is evident that the 
system psychologist will then be required to determine what human capabili- 
ties are necessary. He therefore undertakes a task analysis. By this is meant 
that he makes inferences, based upon knowledge of the nature of human 
functioning, concerning what kinds of abilities, skills, and knowledges are 
required in order for a human being to carry out the specified tasks. The 
results of task analysis provide the means for decisions about those individual 
qualities which can be selected, the kinds of performances which can be 
supported by job aids, and a firm basis for the design of individual and team 
training programs. In addition, they provide immediate definitions of the 
various human performances required, and suggest the measures which will 
have to be applied to these performances in the assessment of human capa- 
bilities. 

It is apparent that the technologies of task description and an 


alysis have 
been markedly advanced 


in technique and objectivity since these have 
occurred within the framework of system development. In fact, the contrast 


between these techniques and traditional “job analysis," with its unsystematic 
procedures and poorly defined terminology, is quite striking. The techniques 
described in this chapter have been remarkably successful, on the whole, in 
forecasting human performances required by newly developed systems, 
together with the human capabilities that need to be selected, trained, and 
supported on the job. At the same time, the gaps in scientific knowledge about 
human behavior are all the more pointedly apparent when these techniques 
are applied to systems development. Chief among these, as the chapter makes 
clear, are first, knowledge about dimensions of human functions, and second, 
how these dimensions are affected by the process of human learning as it 
operates in training programs and on the job. 
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TASK DESCRIPTION 
AND ANALYSIS 


Robert B. Miller 


The process of system development generates many kinds of descrip- 
tions—of system purposes and limitations, of hardware characteristics, of 
schedules and costs, of techniques and operating procedures. None of these 
descriptions is so intimately connected with the development of the “man” 
portion of a man-machine system as is the set of documents which contain 
task descriptions: these are the specific statements of all the interactions of 
man with machine and of man with the system environment. Task descriptions 
are developed on the basis of the decisions which determine the assignment 
of functions to man and to machine, and which therefore influence both 
hardware operations and human tasks within the system. Once developed, task 
descriptions serve as a basic reference for all later designs and plans for the 
Personnel subsystem. One of their initial uses is for the purpose of task 
analysis, a process whose results provide data about human functions, which 
in turn are used to determine the characteristics of job aids, training pro- 
grams, and the assessment of performance of the system and its components. 

In this chapter, we shall first describe these purposes of task description 
and analysis and indicate their relation to the process of system development. 
Following this, we shall discuss the general rationale for task description 
Within the framework of system development. Then we turn our attention to 
the nature of task description, the categories of operation with which it deals, 
and the language it employs. Finally, we shall deal with the topic of task 
analysis and the behavioral structure of tasks. Here we shall be particularly 
Concerned with the kinds of questions which this technological enterprise 
Taises for psychological science. 
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PURPOSES OF TASK DESCRIPTION 


[3] 


It is important to differentiate heuristic description from scientific de- 
scription of a set of events. A scientific description generally seeks to describe 
a set of events with variables which are mutually exclusive and have fixed, 
usually quantitative, relationships to each other. The volts, ohms, and amperes 
of electricity are examples, as are the variables of distance, time, and mass in 
mechanics. In contrast, although heuristic descriptions may aspire to the 
rigorous characteristics of scientific description, they may be satisfied with 
much less. A sufficient criterion for a heuristic description is that it aids a job 
or class of jobs to get done. 

Task analysis at present is a heuristic description of activities at the 
functional interface of the human operator and the objects and environments 
with which he interacts. As such, its value is proportional to its utility and 
economy in the design, evaluation, and operation of systems. We must hope 
that fundamental research may eventually provide us with a descriptive system 
for behavior which can provide the factors and measurement operations for 
an equation of all input-output relationships of potential interest in man- 
machine systems. This hope may, however, contain a paradox. One of the 
major uses for the human in systems is as an improviser, to meet contingencies 
the nature of which can be anticipated only in part, and these imperfect 
anticipations of system activities make for imperfect specifications of human 
input-output requirements. Nevertheless, it is a properly pious hope, not only 
that science will be reduced to practice, but that on occasion that practice 
will be reduced to science. 

Having now dispensed with illusions that task descriptions may have to 
meet the canons of scientific rigor, let us examine the purposes which in fact 
do provide canons of utility for behavior description. 


Support for the System Design Enterprise 


The act of design consists of identifying functions and activities to be 
accomplished and of inventing or selecting mechanisms for accomplishing 
these functions and activities (Flagle, Huggins, and Roy, 1960). The progress 
of functions and mechanisms is a two-way affair. A function such as "signal 
amplification" calls for a mechanism. The introduction of this mechanism 
calls, in turn, for the function of maintenance, perhaps of a special kind. Ful- 
filling the maintenance function may in turn call for mechanisms to be 
selected from those available or invented or adapted for special purposes. 

It is particularly desirable during the formative stages of system design 
for the descriptions of functions, activities, and mechanisms to be fluid on the 
one hand, so as not to curtail the free flow of ideas, but on the other hand to 
be capable of being reduced to denotative detail when necessary. Design con- 
structions and decisions are made in the following contexts (Miller, 1954). 
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Human Engineering. Conventionally, human engineering has been 
associated with the design of the displays and controls with which the human 
and the machine plus their environment interact. The description of at least 
a tentative arrangement of displays and controls being used by the human for 
the purposes intended may suggest modifications and improvements of a gen- 
eral or specific nature. Since the construction of tryout hardware is often 
expensive and inevitably time-consuming, descriptions on paper may be pre- 
pared and used as rough "simulations" of actual configurations and events. 
In some kinds of tracking tasks (steering, guiding, aiming, and others) the 
nature of the human control function can be quantified and equations pre- 
pared. What is known about human "transfer functions" (see Chapter 4) can 
then be applied to the design of displays, controls, and their dynamic inter- 
actions, for the tracking tasks. Only rarely, however, does a human's job 
consist only of a straightforward and predictable tracking activity. In fact, 
machines are doing this kind of task more effectively every year as there are 
improvements in the engineering art. 

The identification of human activities permits many questions to be 
asked about the proposed design. "What will happen if ... ?" is the kind of 
question. Attempts to answer this question in terms of a particular design 
configuration may lead to additions, changes, refinements. Some of these 
Changes may be made in the procedure whereby a task is performed, rather 
than in displays or controls. The description of the procedure for performing 
a task is itself a task description if the display and control objects, and their 
indications and actions, are included. f i 

Task description, of course, must be only tentative until the human 
engineering of the task environment has been frozen. At that time, the human 
requirements are relatively fixed if the environmental conditions with which 
the man-machine entity must cope have been thoroughly designated. 


In a large system the question inevitably arises as to what 
tasks will be done by whom. This is the question of job design: the best 
Organization of human tasks into the job requirements of individuals. This 
Patterning will be dictated in part by exigencies of time and space: the same 
man cannot be in two places at the same time. But within these exigencies, 
there may be options that will tend to optimize selection, training, turnover, 
Stand-by, vulnerability, and other factors of operational and economie signifi- 
cance in the system enterprise. Task descriptions provide input information 


for patterning tasks into job design. 


Job Design. 


Selection. The proper source of information leading to selection proce- 
dures for personnel is description of the behavioral requirements. Selection 
May then be based on already-established abilities, or on estimates of amount 
of transfer from previously learned tasks to the subject task; or the selection 
Methods may have to be based on inferences and tests of aptitudes for learn- 
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ing the tasks. An adequate task description may not only provide precision in 
selection methods, but also avoid downright irrelevance due to semantic 
vagueness. For example, a superficial description of a job might lead to the 
requirement of "numerical ability," which in selection testing could be inter- 
preted as ability to perform mathematical operations in which background in 
calculus and algebra seems suited, although in actuality the task might call 
for no more than the ability to quickly memorize six to ten digits and recall 
them while performing other operations—an ability required, for example, in 
a long-distance telephone operator. Task descriptions provide reference infor- 
mation for reducing such ambiguity. 


Training. Task descriptions certainly provide the substance for the 
content of training; in addition, they may suggest the form and sequencing of 
training. They reference the operations to be used in evaluating both the 
training and the traince. 


Objectification of Performance Criteria 


Since the task description is virtually a statement of human performance 
requirements, the description serves as a statement of the task criterion. As 
such it is a fundamental reference for personnel and system actions and deci- 
sions, and for evaluation and prediction of personnel in the System context. By 
being explicit, it may have the advantages of a contract in stating: if Human X 
does all the things listed according to the conditions spelled out, and nothing 
more, he is doing his job (or task) acceptably. This should put the proper 
burden of responsibility on the system designer and his representatives: the 
human-factors engineers and the writer of specifications for the humans. These 
specifications are the task descriptions; they are also performance criteria. 

Actually, this account describes only an ideal state of affairs, and in fact, 
responsibility for design and operation of a System is generally as changeable 
as the shape of Proteus. From a realistic point of view, it is impossible to 
anticipate all contingencies to which a system and its components (including 
humans) may be exposed in operations. Nevertheless, the task description 
should provide at least the variables in the population of operational events 
which should be sampled (representatively, it is to be hoped) for presentation 
to the human component in testing him, as well as a thorough set of output 
characteristics to measure and relate to test inputs, 

The advantage of a good task description is that it is analytic and may 
thus permit subsamples and subtests to be prepared for testing purposes, with 
a consequent gain in exhaustiveness of sampling and efficiency in testing. 
Some of these features will be explained more fully later. 


Common Reference for the Personnel Subsystem 


The personnel subsystem may be considered to consist of all the design 
and operating decisions and activities that are associated with the human 
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component in the system. These have already been individually identified in 
this chapter. They consist of the design of the man-machine-environment inter- 
face (human engineering), procedure design, design of job aids, task and 
job design, selection, training, exercising, and evaluating. (Other factors—such 
as incentive structures, supervisory and monitoring features, job promotion, 
personnel rotation, supply of new personnel—are tentatively excluded from 
this discussion. See Chapter 7.) Definitive statements of task and position 
requirements in the form of task descriptions provide a common, explicit 
reference from which design and operational decisions can be made with some 
hope of consistency. That is. selection, training, and evaluation may be made 
consistent with each other if they are based on a single set of performance 
Objectives anchored in system operation. 

System development and operation is frequently a vast and often a 
sprawling enterprise. There are many offices, agencies, departments, and 
services that add their bits and pieces to the construction and ongoing activity 
of a system. Coordination is practically impossible without common references 
which are made public to all participants. Coordination is made possible by 
documents which specify system requirements and, by explicit implication, 
the requirements of its components—including, of course, the human com- 


ponents—in terms of specific performances. 


THE RELATION OF SYSTEM DESIGN TO TASK DESCRIPTION 


Design is not a rigorous process except at certain stages. It is true that 
cach design decision puts constraints on what further decisions can and must 


be made for a workable and economically feasible system or system com- 


ponent. But a new concept for a mechanism or process may be invented or a 


new discovery made in a natural phenomenon—or old principles may be used 
in a new mechanism. 

It is important to have so 
Sists of, in order to recognize t 
also an inventive process. Invention 


efficient or inefficient, in performing th : 
reason to believe that practices in the design of personnel subsystems is cur- 


rently limited in effectiveness and certainly in efficiency as a means for per- 
Sonnel subsystem design. This is partly true because the nature of it as a 


Subsystem is generally not recognized. 

In subsequent sections, we shall consider why task descriptions are 
important to the design of the personnel subsystem. We shall also see, perhaps 
idealistically, a picture of its formal requirements. The reader is reminded 
that at present task description is not à rigorous procedure with a rigorous 
set of terms and definitions. Actually, it now consists of not much more than 
à general, conceptual format: the substance of this concept is that the “re- 
quirements" for any mechanism can be specified by a description of what its 


me idea about what the design process con- 
hat the design of the personnel subsystem is 
s, of course, may be effective or ineffective, 
eir intended functions. There is ample 
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work responses should be to the stimuli presented to it. The challenge of a 
classification structure for behavior in what we now loosely call “tasks” 
remains before psychologists. The reader is invited to consider this challenge 
and the many-faceted utility of even partial solutions. 

Implicit in the concept of design is innovation. Innovation may extend 
to the perception and organization of patterns of functional "requirements" 
for a system or component, to devices and to operational procedures. The 
fundamental value consideration in choosing a design option is some ratio of 
performance to cost of the mechanism chosen. But “performance” of the 
mechanism may have to be measured by a variety of criteria, many of them 
incompatible with each other (such as speed and accuracy), and costs may 
have to be measured in a variety of ways not entirely compatible with each 
other (such as long-term versus short-term costs). 

Design therefore rarely proceeds by formula, although features of it may 
seem to become "programmable." Changes in engineering technology, while 
opening new possibilities, also create new problems. Changes in the technology 
of behavior (such as may result from improved training techniques) also 
change the pattern of decision options. 

Heuristics is a way of coping inventively or improvisationally with unique 
events, requirements and combinations of ignorance and knowledge. The 
“heuristic man" says: "I will do the best I can under the circumstances with 
what I have, and try to improve as I go along.” In this sense—and applied to 
the context of system design—task description and analysis is heuristic rather 
than rigorous. This is likely to be true, although perhaps to a lesser degree, 
even if we had a task taxonomy constructed along the ideal lines proposed in 
later paragraphs. Each man-machine-environment interface in new systems is 
likely to pose unique problems. Providing a good answer (the “best” answer 
is best only among known and tested design alternatives) will require new 
thinking, originality, imagination, inventiveness. To the extent that old 
methods of description of behavior raise barriers to inventive thinking, any 
standard rubrics should be used tentatively. 

The concept of “inventing” behavior and behavior settings may seem 
strange to the laboratory and classroom-trained psychologist. Nevertheless, this 
is precisely what the systems designer has responsibility for doing, unless he is 
running some comparative studies on design alternatives which someone else 
has already thought up. 


Decision Structure in the Personnel Subsystem 


The development of a design entity may be thought of as a network of 
interrelated decisions (Miller, 1954). Each decision may be characterized by 
a number of response options. For example, there are decisions as to the 
extent to which a particular man-machine system function should be auto- 
mated, or the extent to which to train a complex human behavior requirement 
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by extensive rote training for each combination of operational circumstances 
or to train by problem-solving technique and general concepts and principles. 
This decision is also related to a decision in selection, since training to operate 
by principle, as opposed to rote recall, may require a difference in kind or 
degree of some abilities as opposed to others. 

At present there is no systematic pattern of subsystem design decisions 
laid out in explicit form. Personnel design decisions are generally intuitive, 
segmented, and local. This procedure permits organizational and administra- 
tive flexibility but also evasion of direct responsibility for feedback: if failures 
and difficulties arise, each administrator may point to the person next in line 
as the culprit. The resulting inadequacies tend to prohibit growth in capability 
of the entire design operation. 

Still considering an idealized state of affairs, a proper behavioral taxono- 
my could help to make explicit the structure of alternatives in personnel sub- 
System design. This could increase and make articulate communications and 
Choices in design operations and later in system monitoring. Design decisions 
and terms used in classifying behavior requirements would have direct inter- 
play. The facts are that this ideal state of affairs is very far from being realized, 
and no formal route that will lead to it has been publicly charted. The student 
Of system design may perceive the challenge to conceptualization in breadth 
and depth in this problem arca. The nature of this problem is an example of 
the difference between conventional problems consisting of a dependent vari- 
able with a few independent variables and the "systems type" problem. 


Description through Terms in a Task Taxonomy 


examine the nature of an ideal method for identifying 
aviors that comprise tasks. It is ideal in that it is 
eachable, reliable with respect to different 
tation in coordinating and directing 


Let us now briefly 
and describing tasks or beh 
Standardized, efficient, directly t 
Observers, and requiring minimum interpre 
action in each facet of the personnel subsystem. 

A taxonomy differs from a mere glossary of terms. Terms used in a 


taxonomy are formally related to each other by explicit principles of inclusion 
and exclusion, The terms are systematically related to each other. They are 
also related to each other by concepts about the operations denoted by the 
terms. In biology, the taxonomic terms “photosynthesis” and “carbon syn- 
thesis” provide descriptive and analytic means of differentiating life forms. 
Note how much more significant these terms are than “animal life” and “plant 
life”: they point to a major basis for relating and applying knowledge as well 
às making an observation and test of differences between "plants" and “ani- 
mals.” Note also that these terms need not be mutually exclusive with respect 
lo an organism in order to be highly useful. There is some evidence, for 
€xample, that some viruses may live by some combination of photosynthesis 


and carbon reduction. 
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For purposes of behavior description, we would ideally have a set of 
categories that would permit any kind of stimulus-response relationships which 
could be found in tasks of any kind to be readily identified and labeled. Cer- 
tainly the total set of these terms should be exhaustive to the extent that any 
kind of behavior requirement can be specified by one or a combination of 
terms in the taxonomy. Mutual exclusiveness of terms may be, from an opera- 
tional standpoint, a vainly sought objective for a taxonomy. 

In addition to simplicity in attaching categorical labels to features of 
behavioral events (or behavioral requirements), each term in an ideal taxono- 
my would point to design operations for efficiently controlling the behavioral 
phenomena denoted by the term. Let us for example, assume that the term 
“visual scanning" appeared as a term in the taxonomy. Having this term in 
the ideal taxonomy would imply that there would be a body of knowledge 
about selection factors relating to this behavior; and also that there would be 
a body of knowledge on efficient and effective training and evaluative devices, 
operations, and sequences of operations particularly appropriate to *'visual 
scanning." In a taxonomic sense, the term "visual scanning" might have 
further value in that it should point to relevant subcategorizations such as 
“scanning through visual noise" where a subbranch of knowledge could be 
brought to bear on human engineering, training, and so forth. 

It can now be seen that an ideal behavior taxonomy would, on the one 
hand, permit the use of terms to identify and label behavioral events and, on 
the other hand, point to a body of design knowledge about the effective 
development, control, measurement and prediction of the behavior in systems 
contexts. Since there are some developments along the line of a selection tax- 
onomy, the need might be greater for the development of a taxonomic struc- 
ture for particular use in training and transfer of training. The classification 
rubrics would permit determining similarities and differences among tasks 
used for research in the learning literature. On these grounds one could 
ascertain the task conditions under which the given research findings were 
applicable and not applicable. The glossary of terms of the behavior taxonomy 
would thus become the relevant index for determining applicable research 
findings and principles for training. 

Presumably, a taxonomy would also provide guidance to researchers for 
setting up and describing their experimental studies. The scientific advantage 
of a taxonomy of this kind for the development and interpretation of research 
data should be obvious. 

A few researchers are currently becoming interested in this problem. It is 
likely to defy conventional approaches which subject small range samplings 
of data to intricate statistical meshes and which disregard the criterion of 
utility of the taxonomy for training and human-engineering decisions. lt is, 
in fact, likely that a breakthrough of some Kind in research technology may 
be required in order to move effectively toward a task taxonomy, just as a 
new research technology is perccived as a necessity for the development of a 
systems theory and systems taxonomy. 
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The reader has now been exposed to the challenge for the future in task 
description and task analysis. Let us forthwith return to the limping practices 
of the present. 


SYSTEM OPERATIONS AND TASK DESCRIPTION 


[3] 


The description of human tasks in a system has a fundamental relation 
to the purposes for which the system is designed in the first place. Just as 
these system goals lead to the statement of hardware characteristics, to an 
equivalent degree they determine the characteristics of the personnel who 
interact in various ways with this hardware. The description of human tasks 
must therefore be carried out within the context of the major categories of 


functions associated with the expected operation of the system. 
There are obviously various levels at which the description of the total 


operations of a system may be stated. As one progresses from the general to 
the more specific identification of system functions, he comes progressively 
closer to the level at which individual man-machine interactions can profitably 
be described, Generally speaking, this point is reached at what we shall call 
the subsystem level, in which one or more human individuals are engaged in 
the performance of some distinguishable portion of the system’s purposes, a 
portion which could conceivably be observed as a separate entity. 

Let us consider, for example, the levels of operational description which 


might be applied to a strategic missile system. 


s level one can distinguish the broad categories of func- 
e that the system will fulfill the purposes for 
e categories, a sample of which is shown in 
the formation of organizational charts and 
ninistration is the name given to the activi- 
ties which govern the flow of information and decisions from one part of a 
system to another, and between a given system and other systems. Operations 
is the branch of system activities specialized for carrying out the primary 
Missions for which the system was designed. It is a mistake to think, however, 
that operations in this specific sense is synonymous with total system function- 
ing; instead, such operations are merely part of the story when viewed at this 
level. Maintenance refers to the prevention and repair of any form of hardware 
malfunction. Although treated as a distinct function, it should be integrated 
during the design conceptualization with system operations, since the time lost 
On account of maintenance often penalizes the mission effectiveness of a sys- 
tem, or may jeopardize it altogether. Logistics deals with the problem of 
supply and demand of materials within the system. The problems of inventory, 
Procurement, and transportation may be more significant to system success, at 
least in the long run, than many refinements in design for operations and 


Maintenance. 


Level 1. At thi 
tions which are necessary to insur 
which it has been designed. Thes 
Figure 6.1, are traditional even to 
assignment of responsibilities. Adr 
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Other systems 


Research and 
development 


System 
organizations 


Operations Maintenance | Logistics 


Etc.-~ 


Denans Intelligence | piei Communications mm 
of an 
organization 
Mission Ground Fuel 
| planning control | Storage Records 
Figure 6.1. Example of organization and department structure in 


systems. 


Level 2. So far, the kinds of system functions described may be applied 
to almost any system, regardless of its purpose. The next level of specificity of 
description, however, brings us again to the purposes of a strategic missile 
system. Each of the general categories of Level 1 can now be seen to have a 
number of subordinate goals in fulfilling the mission. For example, Operations 
might include such functions as range safety, fuel storage, missile preparation, 
and missile launch. Such categories might be called departments, and it will be 
recognized that they can be distinguished for each of the categories of Level 1 
(see Figure 6.1). However, the description of human tasks can seldom if ever 
begin at this level, since the variety and number of people and equipment 
involved are too great. 


Level 3. If one now considers a particular department of operation, 
such as missile preparation, it will be apparent that this can be divided into a 
number of subcategories such as engine checkout, guidance system prepara- 
tion, and so on. In other words, one has now arrived at the subsystem level, 
in which human individuals or teams interact with equipment in a manner 
which makes possible the identification of a unified, subordinate function 
which contributes to the total operation of the system. It is at this level that 
one can begin to describe the specific interactions among men and machines 
that constitute task descriptions. It is essential to note, however, that such task 
descriptions are made with full cognizance of the ways in which they con- 
tribute to the system's goals, as these are conceived at the progressively more 
general levels of description which we have just identified. 
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The Difference Between Task Description and Task Analysis 


A task description may best be understood as a statement of require- 
ments. Although the operations of producing a good task description may be 
substantially aided by a task analysis, we can differentiate between the objec- 
tives of a requirements statement and an analysis statement. 

Task description specifies along a time scale the cues which the human 
should perceive in the task environment and the related responses which the 
human should make in his task environment. For example: "Must run one 
mile in ten minutes on straight, level, and smooth concrete roadway, with no 
wind blowing on a sunny day at 70? F., carrying 30-pound load in addition to 
wearing 2 pounds of summer clothing and 215 pounds of normal GI boots.” 
The word “must” establishes this statement as a job or task "requirement." 
The statement seems extraordinarily complete. 

A task analysis of load-carrying tasks would quickly reveal that the above 
statement is indeed lacking a most critical factor: a description of the load. 
Will it be strapped to and supported on the back or carried in the arms? 
What is its bulk? Its contour and center of gravity? Must it be carried only in 
One position? Clearly it is a different requirement to run with a bucket filled 
to the brim with 30 pounds of corrosive liquid acid than with the same 30 
pounds composed of nuts and bolts in a padded knapsack on a shoulder harness. 

It is possible for a task description to be complete simply by denoting and 
mstances in the stimuli and responses that can occur 
hich tasks may have to be performed. Where 
this is realistic, it is indeed the direct way to establish completeness. But in 
many cases, this would result in huge volumes of documents of description. 
Some analysis of the behavioral aspects of the task will tend to direct the 
task description to more detail about the behaviorally important variables, and 
less detail about variables less critical to successful performance of the task. 
A behavioral understanding (that is, an analysis) of the task requirements 
when viewed in both their physical and psychological settings will therefore 
aid the preparation of task-requirements description in several ways. It will 
Point out what should be described in detail and what does not need detailed 
identification and description for use in the personnel subsystem. Thus there 
Will be completeness of description without undue length. These are the values 
that a psychological understanding of the task to be described can bring to 
the physical description of the task requirements. — 

This issue is more than academic. It has been widely proposed that the 
Preparation of task-requirements descriptions can be performed by laymen, 
more or less as a clerical activity. This is a dangerous part-truth. It is as if a 
layman in engineering were to undertake the performance specifications of an 
electronic component to fit between two electronic interfaces. 

With these warnings in mind, we can now proceed to the superficially 
Simple mechanics of task description as such. We will go into task analysis in 


enumerating all the circu 
in the operational settings in W 
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further depth a little later. For the time being, the description of task require- 
ments is a description of what has to be accomplished in physical terms. 


The Mechanics of Task Description 


Task descriptions can be made at any stage of the development of the 
man-machine interface. They can begin as carly as the conceptualization of 
this interface, and they can be done from blueprints on drawing boards, and 
later from nonfunctional and functional mock-ups. In cach of these cases the 
definition of system requirements (input conditions and output responses) 
must also be available. And, of course, the fewer the details available about 
the physical cues shown and the physical responses required to them, the less 
the detail in the description. The most detailed and obviously the most valid 
description of requirements can be obtained from the task actually being 
performed by sample humans in operational settings. On the other hand, thc 
earlier in system development task information can be generated, the morc 
valuable it is likely to be for system design and timely design of the personnel 
subsystem. 

Like the progressive development of architectural drawings for a new 
building, task description most sensibly proceeds from general "blocking out" 
statements to specifics and details. The following is a summary of steps that 
can be followed in task description procedures. (Compare Miller, 1953b.) 


General Statement of Job and Task Functions. The general functions 
(input-output relationships) assigned to the human component(s) will prob- 
ably have been specified at least loosely by the system designers during their 
layout of what the general functions of the system are to be. They will also 
have determined what classes of components (including humans) will perform 
them. Thus, electronics maintenance consists of the general duties (another 
term for "function") of inspection and checking for in- versus out-of-tolerance 
indications; troubleshooting; replacing; and repairing down to the replaceable 
entity in the machine. 

Another way of getting a bird's-eye view of position requirements is to 
get a dozen or more terms that describe the sequence of major events in a 


"typical" mission or job cycle. Thus, in the operation of vehicles we gen- 
erally have: 


Briefing: the statement of purpose, destination and routings for the mission. 
Inspection of the vehicle. 
Start and warm-up. 
Guiding the vehicle on route to destination. 
Guiding vehicle through traffic. 


Navigation: the proper identification and choice of route indicators. 
Entering into “homing pattern." 
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Debarking. 

Unloading. 

Debriefing, completion of records. 

Each of these activities may be called a segment in the mission or job 
Cycle. A simple version of mission segments is shown in Figure 6.2. 


Briefing, Preparing, Taxiing F lying Breakaway 
flight ber to runway ; to j Bomb front 
plans ircrafi for and initia run target 
mission takeoff point 
Flying to 
Maintaining communications and mission coordination tanker 
rendezvous 
Postflight E Flying to 
inspection| | Taxing : phos eo Holding contact radio 
an Nan a aiii altitude parem base 
debriefing ramp panem i facility 
Figure 6.2. Position segments in a "typical" mission. (Note that a 


variety of contingencies should qualify each box.) 


ditions affecting a mission cycle, or seg- 
ments of it, should then be identified. Will the vehicle be operated at night as 
Well as by daylight, in rain or snow as well as in dry conditions, in fog as well 
as in clear weather? If it is a surface vehicle, will it be driven over rough 
terrain as well as smooth? What varieties of cargo will require special treat- 
ment, and of what kind? This may also be a good time to determine the 
tightest tolerances which will beset successful task performance: the narrowest 
and lowest apertures through which the vehicle will have to be guided for- 
Ward or backward, the tightest curves to be negotiated (also forward and 
backward). Will operation be required through mud, over ice, through 
Shallow water? 
Other continge! 
environment, such as a few suggest 
Occur must be identified, and it mus 


The major environmental con 


icies should be identified. Besides those arising from the 
ed above, the malfunctions which may 
t be determined whether the operator is 
to cope with them or not. (The methods for coping with them should be 
deferred until the details are put into the picture.) Contingencies which may 
be critical to mission success should, of course, come high on the list. Tire 
blowouts on the road, brake failure, steering failure are examples of con- 
Ungencies in the operation of surface vehicles. 


200 - Task Description and Analysis 


Besides environmental contingencies and those arising from machine 
malfunction, there are those brought about by human error. Any mechanism 
which can operate can also malfunction—whether it is a machine or a man. 
At a later stage in task analysis, more careful thought may be given to situa- 
tions likely to lead to error; and information obtained about what provisions 
the human has for detecting his errors and what means will be available to 
him for correcting errors or their consequences. 

The analogues of the factors which guide the preparation of a general 
statement of position requirements as outlined above may be applied to any 
job ranging from cobbler to president of a corporation. The sum of these 
statements should provide a fairly good bird's-eye view of the job. In many 
cases a flow chart of "mission" segments may be drawn, major contingencies 
labeled where appropriate, and time-shared activities indicated by placement 
of boxes along the time baseline. ( Time-shared activities are those performed 
at about the same time, and which have overlap in cues that must be searched 
for, remembered, put with other information, and acted upon when proper.) 
With such a general picture of the total pattern of tasks prepared, details may 
be added to any part without loss of perspective because of preoccupation 
with those details. It is particularly important to identify contingencies (envi- 
ronmental, machine malfunction, human error) because these often do demand 
the highest level of skill and, at the same time, may put the success of the 
system in greatest jeopardy. 


Some Task Varieties. A task is a group of activities that often occur 
in temporal proximity (but not necessarily so) with the same displays and 
controls, and that have a common purpose. A task is a fairly arbitrarily 
bounded set of activities—a rigorous operational definition cannot (and there- 
fore should not) be sought. It is a heuristic term. The general job-cycle state- 
ment guides the selecting and relating of tasks for more detailed description. 

Tasks may be broadly classified into discrete and continuous. A discrete 
(or discontinuous) task is one in which stimulus-response behaviors can be 
marked off into readily differentiated separate steps. On the other hand, the 
general example of a continuous task is that of tracking a moving object such 
as gun aiming, steering a vehicle, and keeping a continuously changing target 
position on a scope centered or matched with a controllable cursor. 

Discrete tasks are of several varieties. One is called standard routine: 
the same series of steps is always followed in the same order. A nonstandard 
routine may consist of a group of steps not always followed in the same order. 
In either kind of routine, the procedure is fixed and specified by step. Non- 
routinized tasks are generally problem-solving in nature. General procedures 
and strategies (principles) may be provided, but the human must also perform 
some logical operations or calculations with the data available to him before 
he is able to make an appropriate response. 

There are two general varieties of tracking task. One is that of aiming or 
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getting coincidence of a cursor and target. In general the objective is to aim 
at the center of the target, or at least some point which may be thought of as 
target center. The device may be either what is called “pursuit” (aiming a gun 
at a flying duck) or “compensatory” (keeping a target airplane in a center 
ring on a radar scope). A second general type of tracking task is that of steer- 
ing vehicles through apertures—for example, guiding a car on a roadway and 
through traffic. In either case, both target and cursor may be moving or one 
of them may be stationary. Somewhat different behavioral principles may 
operate in each of these classes and subclasses of continuous tasks. 
Obviously there will be examples in which the above distinctions between 
continuous and discontinuous tasks will be blurred. After altitude and course 
has been reached in flying an aircraft, many adjustments are procedural, even 


though events are changing continuously. 


Detailed Task Description. Let us assume that a general job cycle 
statement is available to guide the identification of tasks, task clusters, and 
their relationships. The next step is to describe the specific behaviors within 
the tasks. In time-critical tasks (and are there any that are not?), reaction and 
Performance times for stimulus and response should be marked along a 
time baseline. 

The first step is to specify th 
the human that the task is to be p 
of the following: 


e cue or cues, and their context, that tells 
erformed. Then each task activity consists 


An indicator on which the activity-relevant indication appears. 


The indication or cue which calls for a response. 

The control object to be activated. 

The activation or manipulation to be made. 

The indication of response adequacy, or feedback. 

hat provides the activity cue. It may be 


An indicator may be any object tl 
piece of paper on which a message is 


an aircraft instrument or it may be a l sage 
written; it may be the windshield through which one sees to drive. The indica- 


tion, in its broadest sense, is some out-of-tolerance signal which calls for a 
work response—the difference between a present condition and a goal condi- 
tion or information that establishes that difference. The indication may appear 
all at once, or it may have to be assembled by the human by recall through 


Periods of time. 

The control object may be a telephone, S$ 
Activation is any motor response necessary ant : 
demand of the indication and the response requirement to it. 

Indication of response adequacy may be proximal, as by the feel of a 
switch when it is moved into a detent, or distal, as when one hears the motor 
Starting up. In many routinized tasks, the feedback from one step or activity 


is the indication for the next step. 


teering wheel, or another human. 
and sufficient which satisfies the 
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The simplest concept of a behavioral step assimilates several of these 
categories and consists of 


Indication (stimulus) — Activation (response) — Feedback 


In order to get a proper picture of task complexity, two additional kinds of 
information should be provided. One is the kind of disturbance and irrelevance 
(perceptual noise) which can make the indication difficult to detect and 
identify. The other is the number of activities—overt or covert—which the 
human must be performing at about the same time in one or more tasks. 
Describing the task of a big-game hunter would be incomplete if one did not 
specify that the detection and identification of the tracks of the prey will be 
confused by other tracks, grasses, and stones; and that if the hunter is not 
continuously scanning his environment to assure himself that Ae is not the 
hunted, there is severe risk of “mission failure." Generally speaking, the level 
of detail for specifying task activities is about that used in a good manual of 
instructions to a novice. In fact, one of the uses of a good task description is 
precisely that of a procedural manual for the job. Figure 6.3 shows an example 
of task description of routinized activities. 

The preparation of good task descriptions calls for imagination, an 
insatiable ability to ask questions, and an unflagging determination to get (or 
figure out) the answers. It is easy to be content with identifying “normal 
operating conditions” and not think—or take the trouble to find out—about 
the contingencies. It is easy to be overwhelmed by details and ignore the 
possibility that a delicate manipulation may have to be performed under 
time stress in bombardment or in severe acrial turbulence or while wearing 
protective gloves. It is very easy to accept precedent in the new performance of 
various tasks (such as troubleshooting, for example, which is generally done 
in a haphazard, bungling fashion), rather than get to the roots of the 


information-handling operation and expose these for the guidance of train- 
ing method. 


©] 


TASK ANALYSIS AND THE BEHAVIORAL STRUCTURE OF TASKS 


. The task requirements description, as has been pointed out, provides the 
basic reference for the design and development of the personnel subsystem. 
It is the basis, one step removed, for decisions leading to the design of 
selection and classification procedures, training, and evaluation. The step from 
task analysis to specifications for selection and training cannot be made 
directly. Instead, the psychologist must first engage in a process of systematic 
analysis of the behavioral implications posed by the statement of physical task 
requirements. In some cases, these behavioral implications may point to 
deficiencies that must be remedied in the task requirements description by 
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JOB ELEMENT FORM 


Position Line mechanic -- Radar system 
Date 5 May 19 


Duty 1. Adjust system 
[ro] 
TIME ELEMENTS 
: TW REMARKS 
TASK minutes INDICATION Alternatives 
CONTROL ACTIVITY (Include when to do task and/or 
and frequency of task) precautions 


radar 
receiver 
Inverter starts and makes — |Avoid starting 


1.1.1 POWER ON Press 
audible hum, pilot light system with 


Button 
comes on, range indicator |covers 
lights come on, tiltmeter | from DIANA 
pointer comes on scale. rile antes 
personal hazard 
1.1.2 AC voltage Turn AC voltmeter aligns to 
117+4 volts. 


adjustment 
(screwdriver) 


1.1.3 POS regulated Turn POS regulated voltage meter 
voltage indicates 300-5 volts. 
adjustment 
(screwdriver) 


Sweep trace becomes 


1.1.4 BRIGHT Tum 
visible on CRT. 


control knob | clockwise 


Turn as Sweep trace becomes 
required sharper (focused). 


1.1.5 FOCUS 
control knob 


Figure 6.3. Sample of detailed task description. 


Obtaining more information. This systematic study of the behavioral require- 
Ments of tasks is called task analysis. 

A task description of a typist-keypunch operator may include the state- 
Ment: “Must be able to read messages in a variety of handwritings.” The task 
oe iis statement attached to this description might read: “The operator must 
e able to distinguish between those parts of a message in which she can fill 


i à 
n unrecognizable characters from context and those parts of the message in 
ust go back to the source of the 


hi 
hich she cannot depend on context but must 
nterpret from message context 


mes z » HT 
essage for interpretation. Both the ability to ! fror 
t to be trusted require judgment—that is, 


ul ceiling when context is no igual 

this igence of at least average OF above. Errors introduced into the system at 

tr Point will be difficult to correct 1n system operation, and can have catas- 
O9phic consequences if not detected and corrected. 
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Let us use another example. The task description reads in part: "Op- 
erator must back vehicle into areas with no more than 18 feet clearance on 
top and sides." The analysis might read: *This requires absolute identifications 
of size required for the vehicle, and the vehicle may be loaded in different 
ways at different times, so that the operator must be able to memorize these 
different dimensions. Operator will not have complete visibility of aperture 
into which he is backing, since rear of vehicle cuts off view. Operator must 
therefore properly gauge size of aperture before backing into it, and then use 
cues available only on the driver's side of the vehicle for properly steering 
while backing. Differences in illumination will modify nature of cues available, 
hence of aperture judgment and of steering. Task when performed under time 
pressure and fear of discovery by the enemy may generate constant errors in 
judgment and steering, which need to be investigated empirically under differ- 
ent conditions of illumination." The major points of psychological interest 
are in italics. The particular significance to selection and training of the phrase 
"identification of absolute size," as opposed to, say, "sensing of size differ- 
ences" will be evident. 

A task analysis document will tend to provide information parallel to that 
on the task requirements description. Both could form parts of the same docu- 
ment, which might be divided into right- and left-hand sides. Both kinds of 
information, the physically descriptive and the behaviorally analytic, may be 
generated by the same effort—and in fact, this is likely to be an ideal arrange- 
ment, for reasons to be cited later. 

The source information to the task analyst is task requirements infor- 
mation plus all that is known and much that is conjectured in the full area of 
experimental psychology. This is a tall order and invites much randomness. 
The behavior structure of tasks is offered as a guide for organizing a way of 
looking at tasks and task information that is compatible with a way of organ- 
izing knowledge in experimental psychology according to task settings. This 
task structure may also be regarded as an intuitive first pass at a task taX- 
onomy at a very general level. 

If the term "task" is made to apply to a group of activities sufficiently 
extended and complex, that are all surely related to a system goal, we can 
generally find a common structure. If we disregard the specific stimulus- 
response content of a large, randomly selected group of tasks, we will find 
that tasks difler from each other mainly in the relative weighting of the factors 
in this common structure. Except for some additions, the reader will notice 


substantial similarity to the classification Scheme proposed in Chapter 2; 
these parallels are intentional. 


Goal Orientation and Set 


A task is never initiated in a psychological vacuum. There are a number 
of features about the psychological task setting which have rather obvious 
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importance. The first of these is the clarity with which the human operator has 
in mind a set of goal conditions: how thoroughly he knows what is expected 
of him in operational terms. There are some tasks in which a major (but 
often concealed) requirement is to discover a proper set of goal conditions. 
In some respects, this requirement characterizes the inventive situation. 

Where there may be more than one human interacting in a coordinative 
fashion, there may be some occasional uncertainty as to what functions should 
be performed by each. This ambiguity may under some circumstances lead to 
inefficiency; in others, it may provide that flexibility which permits teamwork 
through improvised work arrangements. 

Another factor in the task setting is the motivational context in which 
it occurs. Is the atmosphere “do or die," or is it "business as usual"? Is the 
emphasis in performance on speed or on accuracy? Where there are multiple 
criteria of goodness in task performance, which have priority and under what 
conditions? What is the operator's concept (or what should it be) as to the 
relationship of what he does to subsystem and system performance? Where 
are his attitudinal identifications as to competition and cooperation? 

Finally, what context of stimulus events indicates to him that the task 
must be done, and that it is time to initiate its performance? 

These various conditions may be allowed to develop spontaneously in 
the operator, or some control over them may be undertaken through indoc- 
trination and training. It should be obvious that unless the operator has a 


Clear picture of proper goal conditions, he is in a poor position to profit from 
feedback information—either during training or in operations—about his 
d him toward improvement. The task 


Performance in a way that will ai : 
analyst should have reference to criteria of system performance in order to 
conditions that should initiate a task 


ascertain, directly or through inference, co à 
and the task goals relevant to those conditions. These goals should ideally, of 
COurse, be specified in quantitative terms, and include acceptable tolerance 


ranges, for the variables that enter the task picture. Where possible, relative 
Priorities in task output should be determined in the form of policy and 


Criticality to success of the system's mission. 


Reception of Task Information 

It is of critical importance to identify various requirements associated 
With the reception and interpretation of task-relevant stimuli. Naiveté about 
"the task stimulus" is both common and disastrous among run-of-the-mill task 
analysts, except in the relatively narrow problem area of target tracking ona 
Scope. Because of this common deficiency, considerable attention will be given 
here to problems of identifying task inputs. 
Both early and late in learning to perform tasks, it is 
carch out relevant task cues. This is true 
ookout. It is also true in many types of 


Search and Scan. 
frequently a practical necessity to s 
in vigilance tasks, such as that of ! 
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combat condition, where the operator must not only attend to some ongoing 
set of activities, but must also scan his general environment for surprise attack 
from unexpected quarters. The maintenance technician learns to look at an 
engine and note secondary cues such as frayed or loose wires, burnt terminals, 
fuzz or dust on relay terminals, or bolt heads that have been worn from 
frequent removals. The reconnaissance man—whether looking at natural ter- 
rain, a radarscope, or various kinds of map—develops practices of selective 
searching out of cues. The competent driver occasionally looks at his dash- 
board instruments to be assured that all is well. Where these scanning 
responses require the operator to deviate from a strong primary attention to 
ongoing tasks, this requirement needs special recognition, so that proper atten- 
tion can be given to it at various stages of training. Human-engineering modi- 
fications of the work station may be suggested. The issue may also influence 
selection of operators. The description of the search-and-scan requirements is 
obviously incomplete unless the context of concomitant task activities is also 
identified. 

The search activity in reference here differs psychologically—or at least 
operationally—from the kind of search which accompanies serial decision 
making, such as in troubleshooting. In this latter case, the interpretation of 
one cue leads to a locus of search for the next task-relevant cue. But in the 
case at hand, search and scan are literally a sensory enquiry of a work envi- 
ronment that, to the scanner, may be neutral or not, depending on the presence 
or absence of a task-relevant cue. 


Identification. This consists in perceiving a pattern of cues in such a 
way as to permit it to be named by the perceiver or reacted to by some other 
particular response. The name may be a class name (enemy soldier) or it may 
be a specific name (Joe Jones). The cues perceived may consist of objects, 
symbols, attributes or values of attributes, activities, or conditions. The value 
of an attribute would consist of, say, a speedometer reading of 50 miles pet 
hour. Identification of an activity might consist of the recognition that the 
driver ahead is decelerating. 

The cues that make up an identification may be together or they may be 
separated in time and space. Other conditions being equal, the more they are 
separated, the more difficult the identification may be. This is especially true 
if parts of a pattern must be perceived through periods of time, and especially 
when this time is interspersed with irrelevant cues and other activities. When 
the identification acquires this kind of complexity, it may be difficult to dis- 
tinguish identification from interpretation, which may enter to guide search in 
filling out a cue pattern. 

Verbal listing may be the first of the various kinds of identification 
required by an operator in a task. But since words are often inadequate 
surrogates for physical cues, diagrams and other visual (or sensory) repre- 
sentations should be prepared in order to guide selection and training. One 
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should remember to include the conditions that should lead to the identifi- 
cation of contingencies by the operator, and these in turn should include 
identification of the operator's errors. It is also important to identify the class 
characteristics of noise and disturbances which will interfere with identifica- 
tions during operating conditions. 


Noise Filtering. A signal is information which leads to the selection of 
a response. In a sense, it *throws a selector switch" (but let us not go too far 
with this analogy to human processes). Noise consists of information which 
either incorrectly throws the response switch or obscures the "true and proper" 
signal when it occurs and prevents the throwing of the selector switch. Or it 
may result in the wrong response being selected. 

Much training of the human consists of his learning how to filter these 
disturbances in his task environment. Noise may be environmentally produced, 
as in the case of rain on the windshield, glare of lights at night, static in radio, 
radar, or television. Noise may arise from the sensing equipment itself, as 
through electronic artifacts or malfunctions. And noise may arise from within 
the human himself, as when he is distracted, is motivated by other activities, 
or "projects" incorrect hypotheses about the cues presented to him. 

Stimuli irrelevant to the task at hand, therefore, are potential noise. 
There are some qualifications, however. Studies in perception of radar returns 
have shown that small amounts of relatively homogeneous background noise 
actually improve detection and identification of target information. We recall 
that studies have shown that sensory deprivation (people kept in the dark 
under constant water pressure for hours and days) leads to complete dis- 
orientation, confusion, and hallucination. Studies in the area of attention and 
rapid recognition also support the idea that some kind of sustained stimulus 
(change of energies in the stimulus field) facilitates perception, apparently by 
Maintaining alertness. Speaking physiologically, this is like keeping a neuron’s 


threshold low by a low level energy input not quite adequate for firing it. In 
he elimination of noise as the 


any event, the problem here is not so much t l 
identifying of it and preparing to train for task performance under various 
conditions of disturbance and irrelevance. p 

The complexity of noise signals tends to baffle the descriptive powers of 
the communications engineer in electronics. Because of the much greater com- 
Plexity of the response mechanism in the human, the description of perceptual 
Noise is virtually impossible. Noise that affects human tasks must therefore be 
Coded by class names and denoted by specific examples of the noise phe- 
nomena in task contexts. There seems some reason to hope that the human 
ability to read through environmental noise has $T? degree of generalization 
Within a class of responses. Telegraphic code is an example of what is meant 
here by “class of response." Another response class would be radar returns on 
PPI (Plan Position Indicator) Scopes. Still another would be the human voice 


Speaking a language well-known to the listener. 
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There are two classes of retained information in human tasks. Short-term 
retention is like buffer storage in a communications or data processing system. 
It is information peculiar to a single (task) cycle of operation which must be 
retained for seconds, minutes, or even hours, so that it can be combined with 
other information and used at the proper time to effect appropriate response. 
Long-term retention, on the other hand, is like a set of instructions that is 
"programmed" into the computer and is valid over any and all cycles of the 
task in operations. There is reason to believe that there is at least some 
psychological discontinuity between short-term retention as here defined and 
long-term retention. 


Short-term Retention. "This factor has sometimes been confused with 
“memory span," but this is far too restricted a context for our purposes in 
task description. The waiter's feat in memorizing the order of several patrons 
while they are speaking and later serving each with the dish he ordered, is an 
example of short-term retention. So is the motorist's effort in trying to nego- 
tiate a route just described to him by a passerby. The player's recollection of 
the designation and position of each card played in a hand of bridge is 
another example. 

Skill in virtually all human activities includes some special classes of 
short-term retention. The typist learns to read a phrase or a line ahead of her 
fingers; the motorist memorizes the configuration of traffic around him while 
maneuvering through a busy street; the administrator retains information 
about half a dozen variables each about several people being considered for a 
promotion. By whatever means this ability comes about in a given task con- 
text, there seems clear evidence of improvement by the learner, from “novice” 
to "old pro." 

Far less seems known about the acquisition, retention, and transfer of 
abilities in short-term retention than its importance to behavioral economy 
suggests. It seems clear that a significant factor in skill at decision making i$ 
the range and precision of short-term memory. 

It is difficult to classify kinds of short-term memory with any hope of 
utility. The task analyst must therefore depend on being fairly specific in 
tabulating the kinds and amount of short-term information which must be 
retained in task cycles and job cycles. He should also try to spot “confusion- 
likely" situations that may arise in task contexts. 


Long-term Retention. Long-term retention includes the identification, 
nomenclature, and background knowledge for interpretations of perceptual 
information. The present context of this chapter, however, focuses on the 
retention of procedures. A procedure may be retained in one or a combi- 
nation of several forms. It may be retained as a set of verbalizations, more or 
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less in the form of sentences. The first few lessons in starting an airplane 
would consist of memorizing verbal rules. In order to put the verbal rule or 
statement into practice, the operator must be able to identify the task stimulus 
from its verbal surrogate and must be able to generalize and perform the task 
response from the verbal surrogate of that response. 

Tasks may also be performed by recall of general knowledge and con- 
cepts and by making deductive inferences from them. This is generally the case 
in “problem-solving” activities, of which troubleshooting with limited job aids 
is an example. Activity is guided by strategy (general rules) rather than by 
specific rules applicable to individual stimulus-and-response identities. 

With sufficient repetitive practice, the symbolic mediators between envi- 
ronmental stimulus and overt response will tend to drop out from perform- 


ance. Such non-mediated performance can conveniently be called *automa- 


tized response." Automatization is likely always to be a matter of degree. 
bers may be performed 


Larger and larger clumps of stimulus-response mem 
without mediators, although the clump may be initiated by a symbolic activity. 
In highly skilled performance, this symbolic initiator may be little more than 
the concept of a particular goal state. When we decide to get a drink of water 
in our own home, the concept of getting it may initiate a train of “automatic” 
Tesponses in rising, walking, selecting a glass, turning on the faucet, and 


Tunning water into the glass. 


These distinctions among the ways in which stimulus-response associa- 


tions are encoded in long-term retention have more than academic significance. 
If activities are to be time-shared, then it is clear that some of them must be 
to some degree automatized. One cannot think of the words in two sentences 
at the same time; nevertheless, one can drive a car and converse at the same 
time. If the range of contingencies which can beset a mechanism or a task 
environment is very large, it may be more efficient to teach by general concept 


rather than by specific procedures for each contingency. By acquiring a con- 
cept of the sequence of events in the mechanisms of an aircraft engine, it 
becomes somewhat possible to deduce what to do in a variety of situations. 


The operator recalls cause and effect in the action linkages. | 

But where high reliability of behavior is required, and it is feasible to do 
So, a straight listing of procedures which can be performed with. or without 
the aid of a check list may be the most effective method of instruction with the 
goal of learning a verbally mediated set of operations. This may also produce 
à flexibility somewhere midway between the stimulus-bound behavior in 
automated response and the uncertainty of deduced response. 

Where high reliability of infrequently practiced tasks may be required— 
Such as is the case in a large range of potential emergency conditions—some 
Of all three classes of S-R encoding may be attempted. This would provide 
“associative redundancy,” a variety of recall routes and mnemonics which, 
taken together, would tend to increase probability of recall of the correct mode 


of response to the unusual circumstances. 
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Here is a situation in which the task analyst, by specifying modes of 
response, clearly points to the nced for getting more information about "the 
system" than is necessarily given by the bald task requirements. He is suggest- 
ing not only what must be done but, behaviorally, how it is to be done. His 
knowledge about the task should put him in a better position to exercise these 
insights than personnel people down the line who might receive only "task 
requirements" information. He must remember that the task analyst must 
frequently anticipate many of the circumstances of task behavior, rather than 
observe them. He is working during relatively early stages in the design of 
the total system, perhaps even before hardware is completed, and generally 
before it is fully operational. 

A method of describing a procedure has been previously outlined (see 
page 201), applicable to routinized procedures. Where a procedure has a 
large variety of conditions and alternatives (“subroutines,” in computer lan- 


guage) it may be desirable to consider treating the task as a problem-solving 
task. 


It should be clear from the discussion on long-term retention that the 
description of task requirements should take into account not only what must 
be retained by the operator, but the conditions which may prevail at the time 
that recall is required. Reports and observations have shown in a large variety 
of jobs and tasks in the operational world that the incumbents were seriously 
deficient in verbalizing and performing the procedures both central and 
peripheral to their responsibilities. 


Memory for Codes. A special but important form of long-term memory 
applies to human coding and decoding operations. These operations consist of 
à direct translation of symbols into other symbols in a one-to-one relationship, 
where speed and accuracy are indicators of skill. Typing is an example of 
translating written or typed messages into typescript through manual outputs. 
In this case, the output symbols may be individually identical to the input 
symbols, but the arrangement of the symbols on the source document may 
differ from the arrangement on the output document. This is likely to be true 
in tabular layout of numerical data. Keypunchers on tabulating and data- 
processing equipment perform similar tasks. The telegraph operator literally 
translates from one code (alphabetic and numeric characters) into another 
(Morse) when he is sending, and again, in reverse, when he is recciving 
telegraphic code. 

The description of task requirements should include some specifications 
on the range and characteristics of the set-up procedures and formats for the 
translation of symbols on source documents to output documents or messages. 
Many errors in keypunching data onto punched cards are attributable to 
inadequate skills of the operator in shifting from one "field" (or category) of 
data to another. When practical, a large number of representative patterns of 
symbols should be provided for training purposes. Words and phrases are 
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patterns of alphabetic symbols, for example, and the highly skilled operator 
tends to apprehend symbols in such patterns and "emits" them as patterns. 
Apprehending information in patterns has the value of redundancy (*mean- 
ingfulness"), hence greater reliability, in perceiving the message (as well as 
the symbols) and in translating the message. Even numeric data that ordinarily 
are more nearly like "nonsense" symbols and symbol patterns may acquire 
some degree of context, and both human engineering and training should 
Capitalize on such contexts. Needless to say, severe requirements of speed and 
skill in shifting rapidly from one format to another require several levels of 
“automatization” between the stimulus code terms presented to the operator 
and the emitting of the responses of translation. This automatization will cer- 


tainly be necessary where the operator must exercise a higher order of atten- 
tion on the data he is transmitting, as when he has to deal with the meaning of 
the symbols as well as with their translation. Tasks required of some air-traffic 
control jobs require this dual type of performance. Human engineers are aware 
of some of the dangers that arise in task combinations that require continuous 
automatic responses by the operator as well as his vigilance and “thinking.” 
It seems quite possible that training can have its content and organization 
directed at least toward “face validity” in such dual skills. The vast increase 
and proliferation of data processing will call for increasing numbers of indi- 


Viduals with these skills. 


Interpretation and Problem Solving 
some context of information to bear in 


addition to what is directly presented at the moment to the senses. In this 
Context, the information immediately presented to the senses consists of 
“symbols” from which the interpretation (meaning) is established. Thus the 
visual shapes of the word CAT and an open switch before an onrushing 
train are both symbols. The problem of the task analyst is to specify that 
“context of information” which the operator should have which will be 
Necessary and at least sufficient for the operator in establishing the meaning 
Or in i 

E re modi of any sign are practically infinite. Which one will be 
chosen? This is indeed a critical question. The answer is direct. The infor- 
mation context that the operator should have in his head (or otherwise 
directly available) is that which will enable him to select or contrive the 
Course of action essential to his assigned task responsibility when he is 
Presented with the sign (or symbol) and sign context. If we bear in mind that 
task performance implies goal-directed activity, then, in its rawest terms, the 
relevant meaning of a stimulus consists of the responses or response classes 
Permitted and required of task performance (compare Chapter 2). 

For example, a railroad yardman should probably perecive an open 
Switch in front of an oncoming train with the following "meaning : "Close the 


Interpretation requires bringing 
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switch immediately," or "Stop the train immediately," or *Reroute the train 
immediately," whichever is most practicable at the time. lf none of these 
primary task responses is practicable, another *meaning" may come into play: 
“There will be a train wreck, hence injuries." This raises another response 
option: “Call out the ambulances.” 

Notice that if the open switch were interpreted only as "train wreck,” the 
response directly linked to this interpretation would be the calling out of the 
ambulances. True, the possibility of the train wreck would establish motivation 
to take some avertive action rapidly and correctly, and in human affairs this 
kind of information may not be irrelevant. But, in engineering language, let us 
not confuse the need for a power supply to our control mechanism with the 
design of the control paths and control activations. Precision here permits 
economy of design (in this case of training) and efficiency of operation (lack 
of confusion in performance). 

At least in some limited respects, this analysis of the train-and-open- 
switch situation shows us the intimate relationship between interpretation and 
problem solving. We have defined problem solving as the selection of a task 
response through mediating (thinking) processes. Let us examine the variables 
of our example, schematized in Figure 6.4, in greater detail (Miller and Van 
Cott, 1955). 


Stimulus and Stimulus Variables Directly Presented. The open switch 
is perceived on a given track at a given time. 


Variables of Stimulus Not Directly Presented. Let us assume that the 
yardman has not yet actually perceived the train on the horizon, so that the 
stimulus is not directly presented. He has "knowledge" that the train is due 
at this point in five minutes. The factors which establish the response options 
that may be effective are: distance of train from the open switch, rate at 
which train is approaching, and perhaps the location of other switch points 
on its track. Still another variable may be the location and effectiveness of 
an automatic warning or control signal to the train. 


Classes of Response Options. Response options make up the alternative 
kinds of control options that are available to the operator. In our example, 
they include throwing the open switch, throwing some other switch, signaling 
the train. Within each option class, specific responses, such as which switch 
alternative to select, are available for effecting a favorable outcome in a task 


responsibility. 


Response Implications. In many decision-making situations, the selec- 
tion of an appropriate response alternative may depend in part on knowledge 
which permits conceptualizing the outcome of the response in the context of 
the particular situation. (One might think of this as “response meaningful- 
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Figure 6.4. 


the response has implications for the operator’s own 
future task responsibilities; we have, as an example, a response of “searching 
for more information.” In other cases, the responses have implications for 
effects in the system which may occur beyond the purview of the operator's 
job. Thus, the oncoming train might be shunted onto a siding, but with the 
Probability that a train immediately following would miss the block signal and 
Plow into the first train. Another facet of response implication is the subjective 
Probability estimates of the relative success of one response in achieving a task 
(or system) goal versus some other response 1n the situation. 


Ness.”) In some cases, 


ng frequently requires making some com- 


Goal Priorities. Decision maki dy 1 
]t is important that trains meet 


Promises among several desired outcomes. c 1 
Schedules, but at least up to a point passenger safety is more important. It is 
More important that the express trains rather than local trains be on time. 
(As another example, in defense systems, one might ask whether it is more 
important to avoid shooting down a friendly aircraft because it was not recog- 
Nized than to risk letting an armed enemy get through the defense barrier!) 


Rules for Selecting Responses to Problem Situations. Decision-making 
Tules will tend ito bë generalizations about strategy rather than rigorous 
Specifics, If they are the latter, however, @ quantitative model of the opera- 
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tions, and computers or reference to a table, will be more effective than 
subjective human mental activities. Strategy rules will apply to classes of 
situations and classes of responses rather than to specific cases. 

One broad class of problem solving consists in localizing a cause from 
one or a group of symptoms. This is diagnosis, or troubleshooting. The effi- 
cient solving of this type of problem depends on a strategy of sequential tests 
for information: choosing at each step of inquiry that check or test or inspec- 
tion that yields the greatest relative amount of information about the physical 
or functional location of the object sought. The game of Twenty Questions, 
for example, is generally played most effectively with what is called a “binary 
search" strategy. (This has also been called the “half-split” method of 
troubleshooting.) 

Another broad class of problem solving calls for surveying a situation, 
establishing a goal, making a response, watching for the opponent's response 
in reply, and letting the opponent's reply be in part the stimulus for selecting 
the next response. This is an example of battle tactics; it also applies to busi- 
ness. In general, strategy rules are intended to optimize gains per unit expendi- 
ture of resource. 

Strategy rules, indeed, comprise an important class of information for the 
training of decision-making types of task. They have been elaborately formu- 
lated for card games such as bridge and chess and at some ultrahigh levels of 
the military and government. Rarely do they seem to accompany task require- 
ments information, although the reader may have glimpsed how important they 
are to job behavior. It is easy to imagine and verify the difficulties that arise if 
the operator must, with his often limited resources in logic or specialized 
operations research know-how, develop his own strategies. The task analyst 
may have to accept the responsibility for trying to get answers to the question: 
*What general principles can be used to guide behavior sequences in relatively 
ambiguous task situations?" 

We have seen that decision making depends on various classes of stimulus 
information and of response information, and means by which the operator 
can make appropriate links between stimulus and response. Decision making 
is generally difficult because it requires "holding so much information in the 
head at the same time.” The task analyst may be a key factor in organizing 
the information structure of decision-making requirements so as to make it 
more amenable to training and ultimately to performance in task operations. 
Decision making is almost inevitably required in meeting complex contin- 
gencies in task operations, where any one configuration of circumstances 
happens rarely. Problem solving which produces improvisation and invention 

calls for a treatment outside the context of "task requirements." 


Motor Response Mechanisms 


Although it is often rather meaningless to think about motor responses 
apart from perceptual stimuli, there are some tasks in which such examination 
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can pay off. The pianist's performance is a notable example: he is subjected 
to a variety of exercises and procedures for acquiring sheer skill in dexterity. 
In degree, this is true of all tasks calling for physical manipulation of objects. 
The task analyst may, through suitable examination of the controls of a device, 
anticipate some of the "likely human errors" that will occur. This anticipation, 
ideally accompanied by data, should pinpoint training content that may offset 
these error likelihoods. The human automatization of individual and sequenced 
responses may create both necessary capabilities and some liabilities. Physical 
and psychological stresses may have various effects on motor performance, 


some of which may be partly predictable. 


Implication of Task Structure for Task Analysis 


The foregoing factors in task structure are, in some degree, inevitable 
parts of every task and certainly of every job. Some tasks may have a high 
degree of some one of these factors present; vigilance tasks have a high degree 
of search, scan, and identification, for example. The task analyst can look at 
a set of task requirements and translate their usually molecular statements into 
the molar features of behavior by perceiving them in task structure terms. 
These terms should be more directly translatable into selection procedures, 
training procedures, and human engineering, because the terms are behavioral 
Tather than expressing relations between signals and activations of controls. 
The human component is not a passive wire link, but an active mechanism 
with some built-in properties. By the same token, the collection and organiza- 
üon of task requirements information should be made more useful when 
guided by task structure concepts, as is indicated, for example, by our previous 
discussion of interpretation and problem solving. ] 

Furthermore, there are some bodies of literature about scanning tasks, 
Signal interpretation, noise filtering by the human, decision making, and motor 
Performance. This literature can be used for further inquiry into definition of 

nts information to personnel 


task “requirements,” and also in linking requireme i 3 
decisions in selection and training. The nature of the behavioral “problem 


becomes more apparent. Liabilities in the human component tend to become 
exposed when examined in behavioral terms, especially when we recognize 
that the operator does not perceive and retrieve information like a network of 
wires and switches summoning data pulses from fixed memory cells, but that 
he perceives and recalls through associative networks that are imprecise, 
“noisy,” full of “cross-talk” between recall channels. These features, while 
liabilities in one context, are the basis of his flexibility and trainability. 

. As we shall see in the next section, task structure as described here lends 
itself to the outlining of the training program. Some dimensions of task. per- 
formance omitted from discussion because of considerations of space and the 
Primary objectives of this chapter but which are also of great importance are: 
Physiological and psychological stress; the dynamics of social and team inter- 


dekone 1 "e 
ctions; job and task motivation. 
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Let us recall that task information is not an end in itself. Its purpose is 
to aid in the design and operation of the total system. We may think of the 
human component, once he has been selected, as being “designed” for system 
operation through training. Here we shall consider the implications of task 
analysis for the design of training. 

First, however, we should recapitulate the kinds of information we have 
available. By following the route thus far outlined in this chapter, we have 
obtained a total picture of system requirements. We have a fairly clear but 
continuously unfolding picture of the functional requirements of the system as 
a whole, and of the functional requirements generated within the system by 
the kinds of components we have selected. Within this context of information, 
we have sought out the functional requirements of the human components in 
the system. These are stated in greater or lesser detail by pairings of task 
stimuli—actually or potentially available to the operator—vwith the responses 
and response supports appropriate to those stimuli. We have used system 
information to provide quantifications in the time dimension for stimuli, delays 
required and permissible between stimulus and response, and response rates. 
We have charted the temporal overlaps that may occur for one task with 
another. We have called the sum total of these data task requirements 
information. 

Next, we have examined the behavioral implications of these task 
requirements. In each task and group of time-related tasks we have looked 
for the behavioral implications of search and scan of stimulus fields, identifica- 
tion and interpretation of stimulus patterns, short-term recall, long-term 
memory, decision-making and problem-solving operations, and effector re- 
sponses. This examination was made in the light of what we, as psychologists, 
know about the economics of behavior and the peculiar characteristics of the 
human component before, during, and after training. We have tried to organize 
and analyze task information around knowledge (or at least hypotheses) about 
how the human mechanism organizes the information-processing that goes on 
within itself. The result of this examination of task requirements for their 
behavioral structure and implications is called task analysis. 

The next question is: how do we use information about system require- 
ments, task requirements, and task analyses as guides to training? We will 
want training that is (1) valid—transferable to system operations; (2) com- 
plete—to minimize trial-and-error learning on the job; and (3) efficient— 
in that the training objective is accomplished with minimal cost in resources 
and time. 


System and Task Requirements Data as Performance Criteria 


To the extent that task requirements are specified completely according 
to the operational universe of task inputs and outputs and their time relation- 


—Robert B. Miller - 217 


ships, we have at hand complete performance criteria. Generally speaking, the 
operator has fulfilled training objectives if he can pass these criteria. As train- 
ing specialists, we have therefore a clear target toward which to aim every 
feature of our training enterprise. This in itself constitutes a significant achieve- 
ment in training design. (How these data are sampled and put together for 
Tepresentativeness and economy in building evaluation tests is not within the 
Scope of our present discussion.) And however we may break up the total 
training regimen into pieces of training, we will still have stimulus-response 
Teferences in corresponding portions of the task requirements data. 


Selection and Organization of Parts of Training 


Now, where does one begin? Let us bear in mind that the design of 
training, like the design of any other object or process, is a creative and 
inventive enterprise. The total job can probably be partitioned into training 
Segments in a variety of ways, all of them equally effective in supporting each 
Other, Take for examples the operator of a vehicle, a member of an assembling 
Crew of a missile, an electronic maintenance technician, a guidance operator, 
and an air traffic controller. These are complex jobs made up of a number of 
Interlocking tasks. ona 

One approach consists of examining what we have called a "mission 
Cycle" or "job cycle" in order to find natural "break points" in the cycle. 
These would break the job into job segments, set off from each other ona 
time baseline. The trainee would be taught all he had to learn within one job 
Segment before he proceeded to training on the next job segment. By another 
approach, an attempt is made to find those knowledges which seem to be 
Common to a number of tasks in the job and to organize training around 
these clumps of knowledge. The same kind of organization would be made of 
clumps of mental and physical skills that exercised these knowledges. The 
latter procedure seems to have better promise of efficiency in learning, at least 
In the earlier phases of total job learning (compare Miller, 1956). 

Let us examine the successive stages and phases of a typical learning 
Situation. say, of an electronics maintenance technician. He has been selected 
for training on the basis of various aptitude tests, but he has no background 


1n electronics. 


Orientation Training 
.. First, we will want to provide a motivational background for the tech- 
Nician in training. He is given a general survey of the functional and physical 
Catures of the system in which his tasks are embedded. He learns the purpose 
the System at large and, more specifically, the functions of the equipment 
is to service, and how cach of the outputs of this equipment contribute to 
© Operation of the system complex. He learns the pU E Y cem anie 
Various kinds of equipment failure or malfunction. He also learns the 
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sources of signal and power input to the equipment, so that he can better 
appreciate, and later anticipate, disturbances of input to the mechanism in his 
charge. He will also get, as part of his orientation training, some general 
concepts on how the equipment does its job and a preliminary picture of 
its physical layout. 

The content of this orientational training obviously comes from informa- 
tion about system requirements and the functional organization of the system. 

The orientation serves not only to get the trainee motivationally "identi- 
fied" with the system as an operating, organic entity. It also gives him a 
general map of the physical and functional context in which his various tasks 
fit. This mental map will, of course, become more specific and concrete as he 
continues training. Nevertheless, it can provide an associative reference for 
relating what might otherwise be random bits and pieces of task learning. If, 
however, the orientation material is too detailed and too extensive, the general 
map he should be learning will be obscured by trivia. 


Nomenclature and Identification of Work Objects and Work Actions 


Words and sentences tend to be meaningful to the extent that they con- 
jure some mental image of the object and operation denoted by the word. 
Before the technician can read books, interpret diagrams, or profit from other 
instructional material, he should learn to recognize and name dials, test points, 
adjustment knobs, power supplies, input signal lines, and so on. He must also 
learn to identify in-tolerance conditions and out-of-tolerance conditions. (We 
might adopt the general definition of an out-of-tolerance condition as any 
stimulus which incites some active task response or change in a task response.) 

In order to profit best from instructional materials about a task, it is 
desirable that identifications pertinent to a task be learned to a fairly high 
degree of reliability. These reference images provide a high degree of “mean- 
ingfulness" to the verbal materials, making them easier to learn and retain, 
and with higher guarantee of generalization from classroom and study hall to 
actual task context. Thus when the student reads: "Turn gain control to 
moderate intensity," he will have a mental picture of the gain control knob and 
an image of a signal of "moderate intensity" on the screen of the scope. 

Similarily, we would teach the meanings of nonverbal symbols that occur 
in the task. Numerical names, graphic symbols, and special codes, such as 
binary numbers, are examples. It is inadequate to provide practice merely to 
bare mastery in learning these identifications. It is difficult for the student to 
learn and link ideas and concepts if he is fumbling with the meanings of terms 
that appear in those concepts. 

As a matter of training technique, we should recognize that the student 
has two sets of differentiations to learn in making identifications. One is the 
differentiation among objects and other task stimuli. The other is differen- 
tiation within the lists of symbols that apply to objects. Furthermore, a two- 
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Way association must be developed: the task stimulus should suggest the 
symbol applied to it; and the task symbol should suggest the task stimulus. 
(“Task stimulus" here refers to an object or a signal emitted by an object.) 

The need to learn nomenclatures suggests that in our task requirements 
description we should not omit communications which the operator should 
send or interpret: these may include informal communications to his team- 


mates on task-related operations. 


Search for Job-Relevant Cues 


A dimension of training that can and should be directly related to the 
learning of nomenclature and identifications is that of search for job-relevant 
Cues. In its simplest terms, this information answers the question of the naive 
Operator: “What tells me that I am supposed to do something?” In the 
asks, of course, job-relevant cues appear from predict- 
times. But as any person who is familiar with real- 
rge number of system difficulties arise because the 
o the grindstone: he is not alert to 
e broken junction, the too-hot chassis, 
f hash marks that precede the 


Toutinized portions of t 
able places at predictable 
life operations knows, a la 
Operator keeps his nose too close t 
Contingencies such as the bare wire, th 
the wisp of smoke from a coil, the pattern o 


full image when a radar screen is turned on. . ] : 
Every job is filled with nonroutinized contingencies, hence it demands 


alertness, This dimension of training should probably begin as early as during 
the learning of nomenclatures and identifications and should continue through- 
Out the rest of training. In tasks that call primarily for monitoring activities, 
Of course, search skills may be formalized through concentrated training. 


Training for Short-term Recall 


Psychologists have reported relatively little about this important class of 
ehavior, We do know from personal experiences that the naive bridge player 


May not be able to recall half a dozen cards in his hand after the hand has 
been played, whereas after a few years of motivated practice at the game, he 
May recall not only all the cards in his own hand, but most of those in the 
ands of the others in the game. The acquisition of skill in short-term recall 
Seems to be somewhat related to the phenomena generally subsumed under 
‘meaningfulness,” but the associative nature of meaningfulness is not itself 


very clear. 


Short-term recall of task-relevant information seems to improve with 


general familiarity about the material to be remembered, and with a fairly 
high degree of practice at observing and recalling such information. The 
information should be presented in task-simulated situations. This requires 
that it be apprehended from the manner in which it is typically presented, and 
recalled under situations which the performance of the task demands. 
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In corrective maintenance, the technician is frequently given one or more 
symptoms of malfunctioning. These symptoms must be noted and written 
down or remembered. They must be borne in mind (recalled) both in search- 
ing for additional symptoms (which also must be recalled) and in making 
inferences about the probable locus of the trouble. Unless he marks them 
down in some way, the technician must also recall the checks he has made 
and, through inferential processes, the portions of the equipment which have 
been cleared from suspicion in narrowing down the cause of the trouble. 
These are only a very few of many more examples of short-term recall 
required in this type of job. Although short-term recall is by no means limited 
to problem-solving activities, it does weigh heavily in them. 

There is good reason to believe that under stress, either emotional or 
from environmental overload, the function of short-term recall deteriorates 
more rapidly than that of long-term recall. Whether this comes about through 
failure to notice the cues or to remember them—or both—is beside the point. 
However, problem-solving abilities may frequently be most highly taxed under 
the stress of coping with contingencies. This observation suggests the impor- 
tance, not only of highly learned and “overlearned” procedures applicable to 
such situations, but also of extensive practice in short-term recall as required 
by the concurrent task requirements of such situations. 

Unexplored, except in popular books on memory aids, are procedures 
for providing through controlled training the assistance of special mnemonics 


and techniques whereby the operator will rehearse and organize short-term 
information. 


Training of Procedures 


It is characteristic that on-the-job studies of proficiency almost inevitably 
show that the majority of job incumbents are deficient in knowledge of both 
standard operating procedures and special or "emergency" procedures. Exami- 
nation of training curricula frequently shows deficiencies in amount of training 
given to procedures and inadequacies in the procedures that are taught. Among 
other reasons for these deficiencies are lack of knowledge by the instructor as 
to what the operating procedures for a given job actually are, and criteria for 
establishing proficiency. As onc might guess, all these deficiencies are espe- 
cially marked in procedures that we have called "contingency operations" in 
this chapter. That is, the contingencies which may besct the performance of 
"routine procedures" are frequently unknown, hence neglected during training. 

The significance of information about system requirements and task 
requirements is clearly evident in specification of training in procedural per- 
formance. A procedure may be loosely defined as a series of steps, where each 
step consists of a stimulus, a response, and a response feedback. 

The teaching of a procedure may begin with "orientation" as to the 
purpose of the procedure in the context of system operation. It may be fol- 
lowed by teaching of the nomenclature and identification of work objects, sym- 
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bols, activities. This enables the student to associate the task stimulus with a 
concept of the stimulus, and the task response with a concept of the task 
response. When these two sets of associations have been established, a con- 
Siderable amount of pretraining can be done at the verbal level, with expecta- 
tion of positive transfer to the task situation. This verbal training can cut down 
on the confusion and trial-and-error with which the student approaches 
learning in the actual task environment. It also permits rehearsal of procedures 
When away from the actual equipment and task environment, as well as some 
degree of anticipation (hence of getting "set") from one activity to another in 
the actual task setting. 

These conceptualizations also permit, in some cases at least, more ex- 
tended delays for corrective feedback about task performance. If the student 
can, through conceptualization, reinstate the stimulus and response of the task, 
he can associate with these concepts the feedback he receives (Miller, 1953a). 
Thus if, as a driver, I have taken a wrong turn and I am later given instruc- 
tions as to where I turned incorrectly, I may on the next occasion take the 
Proper turn if I was able to associate the corrective information with my con- 
Cept of the turning place and my concept of the response I erroneously made. 
It might be sufficient merely to be able to conceptualize the stimulus as well 
as the response I should have made. This kind of conceptualization may 
frequently account for what seems to be “one-trial learning. 

Procedures should, of course, be practiced both symbolically and actually 
ina variety of task settings. Frequently these settings in operations will include 
disturbances (“noise”) or the need to effect time-sharing with other tasks. 
In time-sharing, one task may have to be performed more or less concurrently 
with another task. This requirement imposes, among other things, additional 
Skill in short-term memory for each of the tasks involved. Such practice in 
Various task environments will, of course, tend to guarantee the generalization 
from training to the operational scene. The operational scene itself almost 
inevitably varies somewhat from one task cycle to another. 

In cases where it is highly important to system Success that a procedure 
be performed correctly and within prescribed time limits, or where there may 
be infrequent opportunity for practice on the job, or where the performance 
May have to be under internal or environmental stress, additional steps must 
be taken in training. Additional reliability can be built iita task performance 
by Providing the operator with a variety of "recall dine à —- 

Where flexibility of behavior in the performance of procedures is esired, 
9ne or more of the following conditions should be met. The operator should 

àve "automatized" many of the stimulus-response relationships. He should 
also have had sufficient practice so that he 1s at least fairly adept at handling 
the short-term recall requirements during task performance. The task situation 
Should permit him some degree of anticipation of the next stimulus in the 
action series, so that he is not stimulus-response-bound in time. He should 


have a conceptual reference for his ongoing and oncoming performance. 
The: meaning èf “conceptual” reference deserves some amplification. 
in 5 


222 - Task Description and Analysis 


The novice musician generally depends on "finger memory" for playing a 
composition from memory. But if there are any changes in the pattern of 
muscle action, he may break down. He must play the composition the same 
way every time. The highly accomplished musician "hears" the music inter- 
nally, and although he may be partly guided by muscle memory, he is also 
guided by this implicit *image" or conceptualization of the music. He is able 
to play, at least partly, the composition “by ear.” This permits him flexibility 
in modifying his performance from one time to the next without breakdown, 
and with continuity. 

The same is true, in a slightly different sense, for the accomplished 
driver of an automobile. He drives from a concept of what he wants to have 
happen. If he wants his automobile to move into another lane, he does not 
instruct his arm responses to turn the wheel: the motor responses are geared 
to the concept of the goal condition. Thus he can combine a goal concept of 


accelerating and turning so that a smoothly graded and continuous adjust- 
ment is made. 


Training for Decision Making and Problem Solving 


For purposes of the present discussion, let us define decision making as 
the cognitive processes that lead to the selection of one from among a 
“known” set of response alternatives—where these are known by the decision 
maker. The selection process may include computational and other logical 
operations for combining information. Thus, examples of the decision process 
as defined here would range from (a) the selection of a particular one of a 
group of enemy targets against which to aim a missile to (b) the aiming of 
the missile as a result of calculating azimuth, elevation, and rate for intercept 
with the target. The choice of heads or tails in the toss of a coin; the business- 
man's allocation of budget respectively to the following year's plant operations, 
product research, market research, and promotion and advertising; trouble- 
shooting for the cause of a given gross effect—these are additional examples 
of decision making. 

The information essential to training for decision making has been 
described in a previous section. In summary it consists of: 


The stimulus variables in the problem. 

The response alternatives. 

The variables in the goal conditions and their relative priorities. 
Implications of the exercise of response alternatives. 

Strategy rules for selecting response alternatives in order to optimize goal 
conditions in given stimulus situations. Such rules may include procedures 
for efficiently getting additional information towards an ultimate decision. 
6. Allowable time between the situation arising and control action (compare 
Miller and Van Cott, 1955). 


Pepa BO! Ft 


It should be clear that only when these kinds of information are available 
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can there be effective practice undertaken in decision making relevant to the 
job. In general, this information is almost totally lacking in training situations; 
in many cases the fact that decisions must be made by the operator is only 
fuzzily recognized. The consequence of lacking a picture of the structure of 
the decisions required and of information about stimulus, response, strategy, 
and goal variables is that training degenerates into an attempt to teach “all 
about the situation,” with the hope that personal wisdom and logical improvi- 
Sations by the operator will somehow cope with the job problems. Thus the 
Maintenance technician is taught “all about” the operation of the equipment 
and then thrown, willy-nilly, into troubleshooting, where he may sink or 
swim—as may the system itself. Thanks in general to human adaptability, a 
Ssemirandom midway kind of motion between sinking and swimming is usually 
the result. Large amounts of on-the-job “experiences” are required for even 
minimum competence, where even competence is undefined except by com- 
Parison with other individuals similarly benighted. Extensive backup in system 
time, material, and human resources must support the muddling and errors in 
Such misty cognitive environments. 
It ds be saisons to conclude, however, that all the knowledges and 
rules sufficient to cover all job situations and contingencies can be precisely 
identified and neatly wrapped up in a training package. There is always the 
Possibility that some mechanic’s chance knowledge of the nne used in 
Manufacturing a hairpin could put a machine back in shape and an 
World. The question, however, is really which stimulus and response demands 
are probable and which are critical to mission success, as well as the degrees 
of probability and criticality in a population of missions. Training should 
center around this core of probable and critical events as they bear D the 
man's job, and decreasing values in probability-multiplicd-by-criticalty wies 
€ given training time only after the central training has been conclu : a 
Only if additional time and funds are available for "education" as an altruistic 

Princi i . 
Tus last point of view brings US to the BCE of pese ia 
Whereas we defined (for this chapter) decision making as the Ed Ron 
responses and resources already known to the operator, let us de ine pro lem 
Solving (again for this chapter) a$ the activity of wo cs d 
Tesources and response alternatives beyond those Mad r e - bs 
Operator. Obviously, we can find many examples as den x las » 
distinction between decision making and problem solving peed y et us 
Not be concerned with hairsplitting On definitions or PM ape Pipe 
about whether it is possible "to produce anything iod ps er the sun." : e 
can point to examples in which a problem was solved by a person with a 

new idea.” 

It should be clear that the requi 


SPecified in a task description. Tas 
Tesponse to be made to the stimulus. Whe 


rements for invention cannot be precisely 
k description specifies and denotes the 
re this is done only in general terms 
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of stimulus and response, the reason should be for short-cutting the tedium 
of more detailed description. With this in mind, we must tend rather to educate 
as well as train for inventive acts. A few classical suggestions may be made: 


1. Indoctrinate by concept and example that there are always many levels of 

“causation” for any event. 

Indoctrinate by concept and practice to differentiate the function from the 

mechanism that performs the function. 

3. Teach, with examples, the variety of classes of mechanisms and physical 
principles that may perform a function. 

. Teach synonyms (in the form of a thesaurus) for function names. 

. Teach, by any means available, contemplation of a problem situation. 

Teach criteria and, by practice, balance between contemplation that pro- 

vides analysis and hypotheses and the testing of hypotheses. 

7. Indoctrinate the motivation to search for alternative modes of response to 

system situations. 
8. Indoctrinate that a new method of control is not a sufficient reason for 


adopting the method. It must improve over an existing ratio of perform- 
ance-to-cost. 


N 


Ans 


But aside from formal inventions, the development and operation of a 
system containing humans inevitably is accompanied by considerable informal 
improvisation. This has been vigorously demonstrated by the RAND studies 
of group behavior in complex man-machine tasks (compare Chapter 11). The 
nature of the supervision of the system may tend to reduce such actions, or it 
may be permissive or actively inducive to improvisations. Informal means of 
reallocating task inputs and even task responsibilities will tend to arise under 
conditions of heavy load and contingencies on a group. Team training should 
be so set up as to encourage a particular group to explore for optimum inter- 
actions. Obviously, individuals and the group should be provided with feed- 
back which will let them perceive what has been successful (and unsuccessful) 
and why. 

Since continuous dynamic changes in a system make it resemble in many 
respects a maturing organism, recognition should be made in system and 
task description of these generally unpredictable changes. They do constitute 
unknown variance in projections of system and task requirements, but this 
uncertainty should not invalidate a best attempt at such projections. Task 
requirement projections tend to guarantee at least a minimally acceptable 
system performance with respect to mission success. Internal optimization of 
resources by system components then becomes a bonus in reliability over a 
somewhat wider range of potential catastrophes. 

A significant, if not determining, variable in the successful maturation of 
a system is the nature of its leadership and human supervision. This topic is 
outside the scope of the present book. Nevertheless, insofar as this leadership 

may influence the extent of improvisation by individuals and groups within 
the system, the topic should at least be mentioned. Even where there is a high 
degree of automation in a system complex, the quality of human leadership 
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must remain a critical factor in system reliability. Maintenance and logistic 
operations must be accomplished by people who think, plan, and make deci- 
sions, and do so quickly and alertly or sluggishly and ineptly. At least in the 
long run, these are functions of leadership. Unfortunately, we seem to be 
learning much more about job design, selection, and training of people for 
man-machine tasks than for patterns of specific supervisory responsibilities. 
We can hope that studies in man-machine system laboratories will provide us 
with guiding data and principles in this direction, too. 


Training for Motor Response 


We have discussed training for perceptual, procedural, decision-making 
and problem-solving behavior. We turn, finally, to motor response and some 
of its properties that should influence training (compare Miller, 1953a). 

The task analysis, rather than the description of task requirements, 
should be of major help. The analysis should have identified likely human 
errors as a function of the response mechanisms. For example, study of the 
task may reveal operations that require the location and activation of controls 

cues. Special patterns of motor coordination 


by kinesthetic, rather than visual, 
may be shown to be necessary. Occasional reversals of customary actions may 
from an upside-down position. 


be required, such as starting a nut on à screw 
Sequences of motor response may reveal error tendencies in the task. Re- 


Sponses may have to be made immediately following a response which cul- 
minates a difficult task: for example, shutting off the ignition after maneuvering 


a particularly difficult crash landing. 


The possibilities of negative transfer effects from previous learning of a 


Teversed control or the failure to conform to a "population stereotype" may 
be recognized as a basis for special training. And a study of the motor 
demands and the task requirements in context may show what response 
groupings or patterns should be learned and "automated" as a performance 
unit. Where timing of the motor response is critical, special exercise material 
for anticipation, attack, and release of the body movement may be prepared. 

These are examples of what inquiry of the motor requirements of tasks 
may reveal for training purposes. They are certainly not exhaustive even of 
the classes of motor problem that may be investigated. 


Organization of Training Content 


Obviously one cannot teach or learn everything at once. The question then 
becomes one of finding the sequence of the content of training. The organiza- 
tion of training into units, segments, and part-tasks, probably cannot be put 
into formal prescription. The topics identified above do suggest an approach 
to organization and sequence of content at least at the task level. In review 


these consist of: 
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1. System and task requirements data as performance criteria, This is orien- 
tation as to the goals of task performance and what the task is all about. 

2. Orientation. A general functional flow-chart of the operations supported 
by task performance and the mechanisms on which it is performed. 

3. Nomenclature and identification of work objects and work actions. Relat- 
ing the terms, symbols and work objects to each other so that the student 
can read and execute verbal instructions, understand feedback, and com- 
municate his actions to others. 

4. Search for job-relevant cues. Techniques for paying attention to his job 
environment for cues that indicate he should take some kind of action. 

5. Short-term recall. Learning to remember task-relevant items of informa- 
tion peculiar to each task or job cycle, or between the perception of 
stimulus units and motor response. 

6. Procedures. Acquiring relatively invariant associations between stimulus 
events and response actions. 

7. Decision making and problem solving. Patterns of stimulus variables, 
response alternatives, goal variables and priorities, and strategy rules for 
selecting responses in decision making and for inventing or improvising 
new responses in the form of concepts, mechanisms, or operations in the 
course of problem solving. 


8. Motor response. Efficiency and reliability in the mechanisms of execution 
of stimulus-response patterns. 


Except for short-term recall and motor response, we have in this list at 
least a reasonable order for teaching a task, including its contingencies. The 
two items named do not fit into any special place in the order of training, 
and their position in sequence is likely to be highly dependent on the nature 
of the task. But the remainder of this general order seems best to fit the 
requirement for efficient training that emphasizes building upon what is 
already known by the student. Nevertheless, pending considerably more 
programmatic research in task learning, the foregoing recommendations should 
be regarded as tentative, although psychologically reasonable. 


[t] 


TASK DESCRIPTION AND PERFORMANCE EVALUATION 


Evaluation may serve two related purposes. One is to establish the 
readiness of the component to be put into the operational entity we have called 
a system. The other is to predict the level of performance of the system itself 
with the component in it. Level of performance includes, of course, reliability 
of performance across the totality of events and contingencies that will act on 
the system throughout its career. 

Evaluation is generally expensive, both in time and material resources; 
system evaluation is almost overwhelmingly expensive. Therefore it is usually 
possible to obtain only a relatively small number of samples with a limited 
number of variations among hundreds, and possibly hundreds of thousands, 
of variables potentially capable of interacting with each other. To many engi- 
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neers, the uncertainty of performance as it affects error probability of the 
human component seems prodigious. In some respects this is true. In others, 
the human capacity for task adaptiveness (or should we say, the capacity of 
some humans for task adaptiveness?) is also prodigious. In any event, the 
evaluation of human performance poses special problems in sampling of 
stimulus events in order to predict system performance in some kind of 
probabilistic structure. 

The combination of task requirements and task analysis should reveal 
how to sample situations so that performance measurements can be most pre- 
cisely taken and generalized. A relatively large number of expected "trouble 
spots” in performance may be sampled in exercises with the human component 
in local (nonsystem) evaluations. Large numbers of samples are less expensive 
to get and easier to control if restricted to such local exercises. Performance 
in this range of sample problems can then be measured. Subsamples of prob- 
lems can then be intelligently selected from these “Jocal tests" and put into 
System exercises Or observed in system exercises. Better extrapolations of 
Systems results to a universe of problems for that system can then be made. 
This is the conventional problem of efficiency in getting representativeness 
and validity. 2. 

The reader should be warned that what may seem a significant local 
Criterion in a portion of a system may turn out to have little importance to 
total system success Or failure. And the converse may also be true. Hence the 
need for total system exercises. expensive though they may be. Paradoxically, 
the total system sometimes performs better than the sum of the performè 
ances of its components measured individually and calculated into system 
Performance. (Perhaps this is because the enemy—or the system environ- 
ment—is not as hostile, clever, Or reliable an opponent as we thought he or 
it would be.) . 

In any event, the selection of exercises for system component evaluation 
is in itself fraught with difficulty and uncertainty. The programming of total 
System exercises is a monstrous task. So is the analysis of the mountains of 
data that ensue. Precision in the identification of a and task requirements 
May s r and guidance to the task. : 

E un ecc pE Drm we have come full circle in the use of 
System and task information. It has guided the development of the system and 


Served as a measuring stick of the system's adequacy. 
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ONCE THE TASKS THAT MEN MUST PERFORM 
and the jobs they must fill within a system have been 
defined, the development process can proceed to iden- 
tify the men to occupy these jobs. The process of 
choosing human beings and categorizing them for the 
channels of individual development which will lead to 
their effective performance of jobs is based upon pro- 
cedures of selection and classification. Task analysis must have previously 
been carried out, to identify the human abilities required for various human 
performances. Selection and classification is begun by designing psychological 
tests to measure these abilities. Such tests have the purpose of predicting the 
facility of learning specific skills in later training programs, as well as ultimate 
effectiveness of human performance on the job. The following chapter de- 
scribes the basic rationale which applies to the design, development, and use 
of these predictors of human performance in systems. 

The predictions of performance undertaken by selection and classifica- 
tion procedures occur within a framework of certain important assumptions. 
First of all, there are some essential differences between men and machines 
which must be taken into account, including such things as man's versatility, 
his temperamental make-up, and his variability. In addition, one must con- 
sider the characteristics of that larger complex of techniques which are used 
to obtain and maintain the people for the system, which is sometimes called 
the personnel subsystem. The administrative features of this subsystem have 
an effect upon the kinds of selection techniques which are appropriate, as 
well as on its mission. In addition, the decisions which have been made 
regarding the variety of jobs, the training, and the measurement of proficiency, 
as parts of the personnel subsystem, will determine the requirements for 
selection and training. 

The technology of selection and classification has been with us for a 
relatively long time and has achieved many successes in connection with both 
military and nonmilitary systems. Despite continual refinement, these tech- 
niques obviously suggest many as yet unanswered questions for psychological 
research. There are perhaps three major areas of research which remain to 
be invaded by new ideas. The first concerns the question of how to derive 
the abilities to be tested by aptitude measures from descriptions of the per- 
formances for which prediction is desired. Related to this is a second and 
equally challenging question of just what is being measured— what exactly 
an ability is, and how it functions in a causal sense to determine behavior. 
Finally, there is the matter of designing ever more complex and intricate 
models for prediction, in order to take into full account more of the variables 
which are now treated as assumptions. This chapter provides a systematic 
basis for consideration of these problems, as well as others, seen from the 

standpoint of a highly useful technology in system development. 
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THE LOGIC OF 
PERSONNEL SELECTION 
AND CLASSIFICATION 


Paul Horst 


ovided many illustrations of the occurrence 
-machine combinations having specified 


goals, which are called subsystems. In addition to these operational units of 
kind of subsystem, which is identified 


ths system, there exists also another ki r 
with the procedures employed to obtain the personnel who constitute the 
binations and to keep them function- 


" à 
"man" part of various man-machine com 
'ng. This is often called the personnel subsystem, and it may be considered 


to have a most important supporting role in the total operation of any system. 

In this chapter we shall be concerned with the personnel subsystem of a 
Man-machine system. This leads us to discuss the decisions that must be made 
aS to which particular persons shall constitute the personnel subsystem and 
also how they shall be distributed or assigned throughout the total system. 
AS a basis for discussing the personnel subsystem, we shall first consider some 
Important respects in which men and machines are different and what impact 
these differences have on the logic of personnel selection and classification. 
Following this, we shall describe the essential properties of the personnel sub- 
System and how it operates. The consideration of these topics will then make 
It possible for us to describe the requirements of the selection and classifica- 
tion model, the variables which it takes into account, and the ways in which 
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it operates to produce the required personnel for a system. Some important 
research questions which arise in connection with such a model will also be 
indicated. 


Purpose of Selection and Classification 


As indicated in earlier chapters, systems are developed to satisfy the 
needs or desires of people. They may be systems for opening cans or for 
mowing lawns. They may be systems for providing transportation, like auto- 
mobiles, or systems for producing the automobiles. They may even be systems 
developed for an entire nation or alliance of nations for the purpose of 
destroying or subjugating another nation or group of nations. Within such 
systems, of course, will be found many component and contributing systems 
varying widely in degree of complexity. Typically, such systems involve the 
interaction of men and machines. In most systems the trend has been for 
machines to take over more and more of the functions of man, and this trend 
bids fair to continue at perhaps an accelerated rate. Thus far, however, the 
over-all and long-term result has not been a progressive reduction of men as 
components of systems. Rather, as machines take over functions currently 
performed by men, new systems emerge, providing new and different functions 
for men within them. Although automation creates temporary dislocations, 
the trend has not been to reduce progressively the man component of systems, 
but rather to change the systems and the specific functions of men within 
them. It can be argued, of course, that increased mechanization and automa- 
tion has reduced the average yearly time spent by men on jobs—that is, within 
systems—as evidenced by shorter hours, shorter work weeks, and longer vaca- 
tions. But while the average time spent by men within systems has declined, 
there has been no systematic decline in the proportion of total available per- 
sons utilized by man-machine systems within this and other industrial coun- 
tries. Such a trend would be reflected in the proportion of unemployed, 
whether forced or otherwise, in the total employable population. Presumably, 
one of the objectives of modern societies is to utilize its total manpower 
resources within the productive systems which in a very real sense are these 
societies. The strength of a society is, in fact, largely a function of the effec- 
tiveness with which its manpower resources are utilized within the systems 
which constitute it. It is probably inherent in the nature of man that the sys- 
tems he generates will continue to incorporate men as essential components 
of these systems. In any case, in the development of systems, jobs are created 
which are performed by men. Other chapters indicate in some detail the sorts 
of things men can do better than machines and vice versa. In the next section 
we discuss some of the essential differences between men and machines. But 
so long as men can do some things within a system better than machines, the 
system, if it is to be maximally efficient, will require a personnel subsystem. 
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This means that men must be selected for the system and classified or dis- 
tributed throughout the system so as to yield maximum congruence between 
the abilities of men and the requirements of the jobs within the system. The 
problems and principles involved in the selection and classification of person- 
nel for the system are discussed in the following sections of this chapter. 


@ SOME DIFFERENCES BETWEEN MAN AND MACHINE 


The subject has been discussed in previous chapters, and we shall con- 
Sider it here only as it bears on the logic of personnel selection and classifica- 
tion. It is instructive in this connection to delineate some of the important 
ways in which men and machines differ. In this way, we shall be able to see 
Clearly the demands which must be made upon a personnel subsystem in 
order that suitable men shall be available to the system as essential parts of a 
Variety of man-machine combinations. 


Functional Characteristics 


Versatility. Although it is possible to design machines which can per- 
form many different kinds of tasks, most men can perform more different 
kinds of tasks than machines can. However, in the development of a system 
One is ordinarily not concerned with the construction of versatile machines; 
Similarly, one does not typically require men to do many different kinds of 
Jobs, even though they may be capable of them. In a different sense, though, 
versatility is a most significant kind of characteristic of the personnel com- 
Ponents of a system. Most men can do some things better than they can do 
Other things, even though they can do a number of things reasonably well and 
Perhaps only a few things not at all well. What is important in developing the 
98ic of personnel selection and classification is that one man can do task a 

etter than tasks b or c, whereas another can do b better, and another can do 
€ better, The personnel selection and classification problem arises in part from 
this lack or correspondence between the relative abilities of men for different 
inds of activities. In the design of machines for a system, the procedure is to 
"sign the machine specifically for what it is supposed to do, ind in general 
there is no concern about how many asks it can perform or how well 


i 
t can perform them. 


other t 


Individual Differences. Another important T: brin € and 
Machines is this: for any particular action to be accomplished, there are great 
individual differences with respect to how effectively a group of men can carry 
OUt a specified performance; whereas machines are usually designed t0 speti 

Cations for a particular purpose, $0 that there will be no great variation 
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among them. For example, in the case of bomber planes such as B-525, little 
variation in performance characteristics may be expected. However, one may 
expect considerable variation in a group of trainees with respect to their 
aptitudes for piloting B-52s. 


Upper Limits of Performance. Another important difference between 
men and machines is that for any particular function there is a definite limit 
to how well or how much of something a person can do, whereas these limi- 
tations are often greatly exceeded by machines. As a matter of fact, one of the 
primary roles of machines is to do better in, or more of, a particular activity 
than men can do. For example, with rare exceptions, men cannot lift 500-Ib. 
weights, whereas machines can be developed to lift many tons. 


Temperamental Factors. Men differ from machines also with respect to 
personality or temperamental characteristics. One may facetiously refer to 
"temperamental" machines, but in general machines do not get angry and 
avoid performing their functions because they do not care to. Unless there 
is some mechanical or organic malfunction in the machine, it will not refuse 
to respond appropriately to its foreman or supervisor who gives it proper 
instructions. Neither will it try to make its operator think that it is doing a 
better job than it really is in order to get promoted to more important func- 
tions. Unless the machine is ill, it will not fail to cooperate with another 
machine simply because it does not like the other machine, or because it 
would prefer to receive from or give to some other kind of machine. These 


kinds of differences create many problems in the selection and classification 
of personnel for man-machine systems. 


Modifiability. Another important difference between men and machines 
is that while both may be modified, the nature of this modifiability requires 
entirely different procedures. It is true that machines within systems are often 
modified so as to change or improve their functioning. It is also true that with 
training and experience the functioning of individuals or men can be modified. 
But it is much more difficult to predict how and to what extent men's func- 
tions can be modified than it is for those of machines. The difficulty in 
predicting the extent to which performance can be modified or improved for 
different individuals creates one of the most troublesome problems in per- 
sonnel selection and classification. We can make machines to specification, 
given the raw materials. It is misleading, however, to press the analogy and 
say that given raw recruits, we can also train or modify them precisely to 
specifications. This, of course, is the purpose of training programs, the effec- 
tiveness of which is dealt with in more detail in another chapter (Chapter 9). 


Checking Specifications. Another important difference between men 
and machines has to do with their specifications. Presumably, in a man- 
machine system, just what the machines are to do has been spelled out in 
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considerable detail. Presumably, also, there are reasonably straightforward 
Procedures for determining how accurately the machines meet these specifica- 
tions. This is true whether one wishes to find out how closely a ballistic mis- 
Sile will find its target ten thousand miles away or to check the washability 
of an army private's work fatigues. 

The problem is far from being as simple in the case of the man compo- 
nent of the man-machine system. Although a task description and analysis 
for the various elements of a man subsystem may be provided in great detail, 
and although there may be a pretty good knowledge of human capacities and 
limitations in general, the problem of determining whether a particular indi- 
Vidual satisfies a specification based on these various analyses is nevertheless 
Not simple. Instruments may exist which may be assumed to measure these 
functions and so give an indication of the potential performance of the indi- 
Vidual in a man-machine complex. But the problem of selecting and classifying 
individuals from a potential pool into the personnel subsystem of a man- 
Machine system is one which, in the current state of our knowledge, requires 
in addition adequate models for testing the validity of the hypotheses and 
Instruments employed. ae . 

. Checking whether individuals meet the specifications before being as- 
Signed to iob activities is only one of two very important related problems. 
he second involves the determination of how well personnel assigned to job 
Activities are actually meeting the specifications of those activities. Estimates 
of how well men are satisfying requirements of the job are called criterion 
Measures. The logic of assessing human performance is considered in some 
detail in a later chapter (Chapter 12). As we develop the model of personnel 
Selection and classification, however, We shall see that an essential part of 
this model consists of assessing human performance within the man-machine 
System. The problems of such assessment are comparable to those of checking 
Specifications of individuals before they are placed in the man-machine € 
We may hypothesize that if individuals satisfy age ane ey 
also satisfy corresponding job requirements. One o Es ems m Sia 
'N verifying hypotheses is to get enough experience on the basis o Ls ic 
reliable conclusions can be drawn. We must have an adequate number o cases 
On the basis or which to validate our logical analyses, and this requirement 
Must be incorporated in the personnel selection and classification model. 
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ones, so that we may have turnover in the machine components, resulting in 
a flow of machines through the system. For any specified system, however, 
the nature of turnover or the flow of these components through the system is 
essentially different with respect to men and machines. In the case of men, 
it is not typical that they leave the system primarily because they wear out. 
Neither is it typical that they leave the system because new and better types 
of men are required; although the latter occasion could occur in the develop- 
ment of an entirely new kind of system, such as man in space. 

Since the personnel selection and classification model requires a flow of 
personnel into and out of the system, let us consider briefly the causes of this 
flow. The outflow from the system or from components of the system may be 
of two kinds, favorable and unfavorable. The unfavorable type of outflow 
may be due to faulty selection in the first place, resulting in persons who must 
be removed from the system. The second type of outflow is that resulting 
from persons who prefer to go to other systems or are more urgently necded 
in other systems. 

The inflow of the system may be of two kinds, the first of these being 
the inflow required to replace the outflow from the system. The second kind 
is that resulting from expansion of the system. System development should 
proceed so that expanded systems may benefit from the experience of smaller 
initial systems. The flow of personnel through a system is an essential feature 
of the personnel selection and classification model. Without taking this into 
consideration, it would be impossible to develop a man-machine system unless 
great reliance were placed on crystal balls or supernatural guidance. 


Supply and Demand. Another practical consideration which bears on 
the theoretical model is that of supply and demand. Within rather wide limits, 
the problems of raw materials from which to produce the machines within a 
system are not crucial. Although raw materials may be limiting factors, if 
more are needed to produce more machines, they usually can be found. But a 
selection and classification model which is based on the assumption that the 
problem is primarily one of sclection and that the resources from which to 
draw are theoretically infinite is doomed to failure in practical application. 

The problem of classification is imposed because of the fact that the total 
available manpower is limited. We must consider how this restricted volume 
of available manpower may be made best to satisfy the needs of the man- 
machine system. Presumably, the requirements of the man-machine system 
will be spelled out; the different kinds of functions and the number of persons 
required for each job will be specified (compare Chapter 6). But for any 
man-machine system, no matter how defined—whether it be a weapon sys- 
tem, a manufacturing system, an educational system, or, for that matter, an 
entire nation—it is most unlikely that the various kinds and degrees of abilities 
required to perform the operations demanded of the system will happen to 
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be just those represented within the personnel pool available for manning the 
Eq This disparity is the basis for the classification model we shall discuss 
ater. 


THE NATURE OF THE PERSONNEL SUBSYSTEM 


[5] 


In developing a logic of personnel selection and classification, several 
aspects of the personnel subsystem are crucial. One of these is its administra- 
tive character; one cannot divorce the model from the administrative frame- 
Work in which it must operate and which imposes restrictions upon it. Second, 
the mission of the system, both with respect to objectives and scope, will have 
a bearing on the way in which a personnel subsystem may be defined. Since 
à central concept of a personnel subsystem is that of jobs to be done, we can- 
Not discuss the characteristics of the system without a consideration of the 
Jobs within it. But the jobs to be done presuppose some kind of requirement 
9f the men who are supposed to do them; so that job requirements are an 
essential feature or characteristic of personnel subsystems. Presumably, also, 
an individual does not just work at a job, but he also works at it with some 
degree of proficiency; thus, as we have indicated previously, the logic of per- 
Sonnel selection and classification involves the concept of proficiency evalua- 
tion. A selection model which provides for verification or validation must 
define the personnel subsystem so that flow of personnel may be assumed to 
characterize it. Another characteristic of the personnel subsystem which the 
Selection and classification model must take into account is the modification 
9f human functioning, which implies a consideration of training programs, 
9rmal or otherwise, "" 

This enumeration does not perhaps include all relevant characteristics 
9f à personnel subsystem, but it should serve as a reasonably satisfactory 


asis for the development of the model. 
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refer to what may be the larger framework within which specific personnel 
subsystems are embedded.) The first of these we shall call a completely auton- 
omous system. The second is an autonomous subsystem within an autonomous 
system. A third type of administrative organism, from the standpoint of a 
selection and classification model, consists of homogeneous groups of autono- 
mous systems. A fourth type consists of homogeneous groups of autonomous 
subsystems. 

The completely autonomous system may be defined as one which for all 
practical purposes operates without restriction except of the most general sort 
imposed from outside, such as may be provided by government controls. The 
typical example of such a system is an industrial organization, such as Gen- 
eral Motors or Procter and Gamble, or a governmental organization, such as 
the Department of Defense or the Treasury Department. In the educational 
field, we have autonomous universities and colleges. Other examples are small 
business organizations or small independent government bureaus, or similar 
enterprises which are relatively small in scope and operation. These last are 
noted because when they are defined as unitary man-machine systems, they 
usually do not individually include a large enough man subsystem to make 
possible the application of an effective selection and classification model. 


Autonomous subsystems may be regarded as departments or divisions of 
a large autonomous system which in themselves may operate with considerable 
autonomy. Examples of such subsystems are the Chevrolet and Buick divi- 
sions within General Motors, or the Army, Navy, and Air Force within the 
Department of Defense. Or we may consider as subsystems of a large national 
industry its distribution and manufacturing divisions. 


Examples of homogeneous groups of autonomous systems are pro- 
vided by cooperative associations of autonomous systems, such as trade or 
professional associations. For example, an association of independent grocers 
within a town would be a homogeneous group of autonomous systems. One 
may consider the high schools within a state cooperating in a state testing 
program as a group of autonomous systems. From the point of view of per- 
sonnel volume, the concept of homogeneous groups of autonomous systems 
is extremely important. It also presents some problems and difficulties from a 
practical point of view. One notable example of a homogeneous group of 
autonomous systems cooperating in the development and application of a 
large-scale personnel selection model is the Life Insurance Agency Research 
Bureau, which represents most of the major life insurance companies in the 
country. Unfortunately for the development of adequate personnel selection 
and classification models, such patterns are very rare in this country. 

The fourth kind of administrative organism, the homogeneous groups of 
autonomous subsystems, is exemplified in a voluntary association of similar 
types of subsystems drawn from larger, completely autonomous systems. Such 
an administrative organism might be represented by association of the ballistic 
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missiles departments of the various Army, Navy, and Air Force groups. An- 
other example might have been (but was not) the voluntary association of 
agencies in the Army Air Forces and the Bureau of Naval Aeronautics during 
World War II, for the purpose of developing personnel selection and classifi- 
cation models for air crew personnel. Such examples are hard to come by, 
Since men, unlike machines, become ego-involved in their own activities and 
jealous of what they regard as outside threats to these. Such differences be- 
tween men and machines may often militate against the structuring of admin- 
istrative organisms to satisfy the requirements of the ideal theoretical person- 
nel selection and classification model. 


The Mission 


Man-machine systems may evolve because of objectives previously estab- 
lished. Sometimes, too, a definition of objectives helps to structure more 
effectively an existing system. Often analyses of man-machine systems may 
reveal unrelated objectives which have developed by accident. For example, 
the general objectives of a small business system may be to manufacture 
grinding mills and household vinegar; accidents of history may combine 
apparently unrelated objectives. Usually, however, in the modern man- 
Machine system the spelling out of general objectives, as in the development 
of specific weapon systems, occurs before the system is set up. But the devel- 
Opment of personnel selection and classification models proceeds from the 
More specific objectives which contribute to the over-all mission. . 

In defining the administrative organism so that it will satisfy the require- 
ments of an efficient selection and classification model, the scope of the mis- 
Sion is also relevant. The organism may be monopolistic, so that we have only 
One completely autonomous system, as in a single ballistic missile man- 
Machine system. If this is the case, then the administrative organism 1s auto- 
Matically defined to include the maximum amount of personnel available within 
4 political or social system for the particular mission. If, however, the mission 
I$ competitive with other systems, such as competing weapon systems within 
the Defense Department or competitive missions among small business organi- 
zations, then it is difficult to establish either homogeneous groups of autono- 
Mous systems or autonomous subsystems to satisfy the requirements of an 
ideal personnel selection and classification model. f 

On the other hand, let us suppose the combined output of a group of 
autonomous systems or subsystems is no more than adequate to meet the 
total need, and that the cooperative effort of homogeneous groups of autono- 
Mous systems or subsystems is recognized as contributing mutually to the 
Objectives of the members. Then we may have the basis for a more adequate 
and logical personnel selection and classification model. The Life Insurance 

gency Research Bureau is one example previously referred to of coopera- 
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Another example which has been regarded by many observers as a most 
unhappy illustration of competitive scope is that of the many competing com- 
mercial organizations who are asked to bid on military contracts involving the 
development of costly systems. This makes for tremendous waste of highly 
technical personnel. It also results in a fragmentation of the man subsystem, 
so that an efficient selection model cannot be applied. 

So much for the implications of the administrative organism for a logical 
selection and classification model. We shall now consider briefly how the jobs 
to be done within a man-machine system relate to the development of such 
a model. 


The Jobs in the Man Subsystem 


As indicated in Chapter 6, it is important to describe jobs in such lan- 
guage that inferences about selection can be made. These descriptions must 
enable us to distinguish between the activities and the objectives of the job. 
It is from the job descriptions that hypotheses for the development or pro- 
curement of appropriate selection instruments are formulated. 

Perhaps one of the most interesting and also difficult aspects of the man 
subsystem has to do with grouping of jobs into homogeneous subgroups. How 
this grouping shall be defined or what shall be the basis of the grouping has 
not been well worked out. However, if those individuals who can function 
satisfactorily on one of two jobs can do so on the other with little or no 
training, and if in general those who cannot function satisfactorily on the one 
cannot do so on the other, then the two jobs may be said to belong to the 
same group. In general, jobs calling for the same functions are regarded as 
similar. Many jobs which have different names and objectives may be similar 
in their requirements, whereas other jobs which have similar names may have 
different requirements. 


Supervision Factors in Jobs. In discussing jobs in terms of supervision 
exercised, one may consider roughly three different levels. These are: no 
supervision, first-line supervision, and executive or administrative supervision. 
The selection and classification models as traditionally formulated are applica- 
ble almost entirely to jobs involving no supervision. Such models, of course, 
ignore the systems approach. It has been common for many years in large 
industrial organizations to select, not only men who will perform the actual 
productive operations of the system, but also those with potential supervisory 
and administrative abilities, who will be scheduled for special programs of 
executive or managerial development. These sorts of abilities and the kinds 
of functions involved in them do not appear to have been as thoroughly inves- 
tigated as some of the simpler individual functions outlined in Chapter 2. 
However, some elaboration of interpretative and inventive functions are 
almost surely involved. 
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In any case, one cannot ignore the administrative functions in man- 
machine systems in discussing the logic of personnel selection and classifica- 
tion. Systematic efforts to distinguish between the types of supervision 
involved in first-line supervision and in the more remote executive and admin- 
istrative types are at best limited and sporadic. Such efforts should be an 
essential part of the analysis of the task in a man-machine system. It is not 
unlikely that malfunctions or failures to achieve mission objectives could fre- 
quently be traced to a neglect of this aspect of a system. For example, a 
technical sergeant may have misread a gauge or failed to check a particular 
Valve and thus caused a mission to fail. On the other hand, if a lieutenant, 
Colonel, or a general is not functioning adequately within the system, even 
though the individual parts of the man-machine activity complex may be oper- 
ating satisfactorily, the mission will very probably fall short of its objectives. 

As one goes up the line of supervision, the problem of applying an ade- 
quate selection and classification model becomes more difficult within a 
narrowly defined man-machine system, because of the relatively small num- 
bers of men involved at these levels. . 

Manpower Requirements. It has been pointed out that the number of 
men required for each particular kind of job is an important item of informa- 
tion for the selection and classification model. The more naive models which 
have been in use for many years have tended to neglect this factor, and have 
assumed in principle that the source from which persons could be Re En 
for any particular job was for practical purposes infinite, so that one did not 
Need to be concerned with how many people were required. Wants War II 
taught clearly, as must any large-scale mobilization program, the existence of 
a close relationship between manpower requirements and sources. . T 

An adequate model must distinguish between classifying men into A 
ferent kinds of jobs and classifying into different levels within a given jo : 
. 9T example, one might classify a group of men into six d aa = 
jobs and then. on the basis of their classification indices, subdivice cach Kin 
9f job according to the level of abilities required for its successful perform- 
ance, For example, let us suppose the jobs are those requiring os 
of Symbols, such as numbers. Appropriate evaluation indices for this function 
May be used to classify the higher group into accountants and Es etia 

9okkeepers. This is, of course, an oversimplification, since other abilities 
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even the vast volumes of psychological tests—achievement, aptitude, per- 
sonality, motor skills, and the like—have not in general been developed from 
theoretical analyses of human functions. It is likely that more fundamental 
approaches, taking their cues from psychological laboratory investigations, 
would be extremely useful in structuring, organizing, and integrating the vast 
and confusing array of arbitrarily labeled measurement and evaluation devices 
which flood the market. 

Frequently, job requirements are spelled out in terms of education and 
experience, as well as in terms of specific psychological or achievement tests. 
From the point of view of the selection and classification models, both educa- 
tion and experience, on the one hand, and aptitude, achievement, and per- 
sonality measures, on the other, may be regarded as providing estimates of 
proficiency in the functions required by specified jobs. Actually, the selection 
and classification model including its validational mechanisms can test the 
adequacy with which these assumptions are satisfied. However, well-defined 
rationales for specifying and evaluating items of education and experience in 
terms of more fundamental human functions are not currently available. 

One important aspect of the role of job requirement specifications has 
to do with the evaluation of personality characteristics. Presumably person- 
ality variables are important for the efficient functioning of a man-machine 
system. They are important because men differ essentially from machines in 
that they do cooperate, they do get angry, they do dislike one another, and 
so forth. But the entire area of personality analysis and evaluation has not yet 


reached the point where it can be articulated satisfactorily with the type of 
analysis set forth in Chapter 2. 


Proficiency 


One of the important activities within a personnel subsystem which must 
be considered in the evolution of the selection and classification model is 
that of proficiency evaluation, or assessment of human performance on the 
job. Without taking into account this aspect of a personnel system one can- 
not have a model which can be validated. Chapter 12 is devoted to the topic 
of assessing human performance. In this chapter we shall discuss the subject 
only briefly in order to relate it to the development of the selection and classi- 
fication model. 


Proficiency Evaluation. One of the characteristics of the selection and 
classification models is that it is concerned with prediction. On the basis of 
behavior previous to selection, predictions are made as to how well individuals 
will function in specified job situations. In spite of continuing violations of 
this principle, it is axiomatic (for anything except a crystal-ball model) that 
we have available, on the same group of individuals, measures of perform- 
ance for a specified set of activities prior to introduction into the man-machine 
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system, and also measures of performance in job activities after introduction 
to the system. One modification of this principle is that the evaluations on the 
variables normally taken prior to participation in the jobs may be taken while 
engaged in job activities. This variation in the model, however, assumes that 
performance on nonjob activities while on the job will provide an adequate 
index of performance on these activities had they been assessed before coming 
On the job. These two types of measurement of nonjob performances as a 
basis for predicting job success have been discussed in various textbooks on 
Personnel selection. Not always, however, have the assumptions been clearly 
stated, and usually they have not been validated. 

Three broad types of evaluation on the job may be employed. One of 
these is in terms of objective measures, such as length of time on the job, 
number of salary increases, current salary, units of production, spoilage, and 
the like. Another kind of measure which may be used is based on objective 
Or achievement tests. A third type of evaluation is that of supervisors' ratings. 
Here many specific methods are available, and this is not the place to describe 
them. The important point is that each of these three types results in some sort 
9f quantitative or numerical value. 

Let us next consider an important aspect of supervisory ratings as it 
Telates to the supervisory framework of a proficiency evaluation system, as 
Well as to the types of administrative organisms which we considered earlier. 

given supervisor can be responsible only for a limited number of persons. 
Ordinarily, such a supervisor cannot be responsible for a group large enough 
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Chapter 12. At any rate, proficiency evaluation must provide feedback to 
the selection and classification subsystem. These evaluations must be readily 
and freely available to the selection subsystem if the model is to include self- 
correcting mechanisms. 


Selection Procedures 


Among the major considerations in typical selection procedures are the 
kinds of instruments used, the basis on which these instruments are used, the 
allocation of responsibility for selection within the organizational structure, 
and, perhaps most important, the relation of the selection procedures to the 
proficiency evaluation program within the system. 


Types of Instruments. The kinds of instruments used may profitably be 
discussed under four major groups: psychological tests and measures of vari- 
ous kinds, biographical inventories, personal references, and interviews. BY 
far the most important of these groups from our point of view is the first. We 
have already referred to worker characteristics and capacities, and have indi- 
cated that a great volume of instruments has been developed to assess these 
various capacities, abilities, and personality traits. We have pointed out that 
the approach has been largely pragmatic and has not followed a comprehensive, 
systematic analysis of human functions. We shall point out here that the 
underlying philosophy of psychological tests—whether these be tests of ability, 
proficiency, aptitude, personality, temperament, or interests—is that they 
constitute specific and presumably relevant kinds of stimulus situations which 
will elicit behaviors considered relevant to job requirements in a man-machine 
system. Measures of this kind are supposed to elicit samples of behavior 
characteristics of the reacting individuals. Implicit in this reasoning is the 
assumption that the sample behaviors are representative of the typical or usual 
ways of reacting to similar stimulus situations, and that therefore these be- 
haviors may be used as a basis for predicting behavior in subsequent similar 
situations. That these assumptions are all too frequently not met in actual 
practice is beside the point. It is a function of the personnel selection and 
classification model to test and revise assumptions of this sort. 

A particular example of this approach is provided by the biographical 
inventory. This is one sort of scatter-gun approach to the sampling of past 
behaviors, stimulus patterns, and environmental influences. The essential char- 
acteristic of the biographical inventory, as of other tests, is that the stimulus 
situations are set up in the form of items with controlled responses and 
methods for evaluating these responses objectively. 

Another common type of selection instrument is the personal reference. 
Actually, personal references can be provided in the form of answers to objec- 
tive questionnaires or they may be given in free-answer verbal form. Perhaps 
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thc type calling for objective responses to a structured questionnaire could 
be used with profit much more extensively than it has been. In general, the 
Purpose in using references is the same as in using tests—namely, to elicit 
samples of behavior. Here, however, the behavior samples are not directly 
those of the individual, but only reactions of other persons to behaviors of 
the individual. In spite of the extensive use of references in personnel selec- 
tion and classification, experimental evidence for the value of such instruments 
is still surprisingly lacking. 

Perhaps the most commonly used selection instrument of all is the inter- 
view. Its use is almost universal in all systems of personnel selection. In the 
interview, the stimulus situation is usually provided in terms of auditory 
Stimulus symbols by the interviewer, to which the interviewee responds. The 
assumption, of course, is that the stimulus symbols are directly related to 
Specific job requirements and that the responses of the interviewee will be 
Predictive of his functioning or responses in a particular situation. The inter- 
view may be highly structured and follow a specified pattern, or it may be 
Completely unstructured. It is not unusual in interview situations for the inter- 
Viewer to do most of the responding, giving the interviewee little opportunity 
to do so. Therefore at the end of the interview the interviewer may have only 
a small and irrelevant sample of behavior from the interviewee on the basis 
of which to make selection and classification decisions or recommendations. 

Perhaps the greatest advantage of the interview as a prediction instru- 
Ment stems from one of the essential differences between men and machines 
Suggested earlier, This is that the effectiveness of the man component in the 
Man-machine system is a function, in part, of his interactions with other peo- 
Ple, whereas this is not the case, except in a superficial mechanical sense, with 
Machines. The interview provides just such a stimulus environment- namely, 
One in which another individual interacts with the interviewee. This charac- 
teristic of the instrument also applies in part to personal references. Objective 
tests, on the other hand, aim to remove personal judgment of the individual 
from the behavior situation. It is true that with some types of testing situa- 
tions, the interaction of the tester and the subject may be regarded as impor- 
tant, Experimental and objective evaluations of such approaches are, however, 
very limited, and in general such results as are available are not strikingly 
Positive, 
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native in practical situations where we are faced with the necessity of pro- 
ducing an operating system as rapidly as possible. It is inevitable in such cases 
that rational bases must play a large role in initiating selection procedures for 
man-machine systems. But it can well be argued that all logical and rational 
approaches must consist of applications or extrapolations of past experience. 

When a new man-machine system is initiated, a great deal of experience 
with personnel may be available from similar types of systems and with ref- 
erence to similar types of analyses in previous situations. Interpretations and 
certain assumptions can be made in taking over in part or in modified form 
procedures which have been verified in similar systems. In this way, bodies 
of quasi-scientific results may be used in the preliminary establishment of 
selection instruments for a newly initiated system. 

Finally, the basis for determining the instruments for selection and classi- 
fication which concerns us most is that based on systematic and structured 


experience according to a prespecified model, in accordance with the proce- 
dures of scientific research. 


Responsibility for Decisions about Selection. One of the characteristics 
of selection procedures which can influence the development of the selection 
model has to do with the allocation of responsibility for critical decisions in 
the selection of personnel. This responsibility can be allocated in one or both 
of two different ways. 

In the first approach, line personnel responsible for the work of a new 
employee may make the final or critical decision in the selection procedure. 
This sort of allocation is common in most industrial organizations. The phi- 
losophy underlying the procedure seems to be that, since a large part of the 
success of an employee will depend on his interaction with his supervisors, 
the supervisory personnel under whom he works must assume final responsi- 
bility for his induction into the man-machine system. 

The other approach to selection and allocation of personnel assumes a 
staff organization or subsystem responsible for the induction and distribution 
of men to the system. The philosophy here appears to be that selection and 
classification procedures are not only sufficiently adequate and accurate but 
also sufficiently specialized and complex, so that a staff division can function 
more effectively in the optimal selection and assignment of personnel than a 
system depending largely on supervisory personnel. This latter type of phi- 
losophy is exemplified in civil-service and military organizations. Many sys- 
tems employ combinations of these two extremes. 


Relation to Proficiency Measurement. An important characteristic of 
selection procedures is the nature of their relation to the proficiency evalua- 
tion program within the organization. A selection and classification program 
can in general be no better or riore effective than the proficiency evaluation 
program within the system. An interesting exception to this statement occurs 
in the case of a system which measures performance of the man subsystem 
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reliably but not validly. It is possible that a proficiency evaluation program 
may measure with great accuracy factors which are largely irrelevant to mis- 
sion objectives. To the extent that selection and classification procedures are 
based on such programs, the system will fall short of achieving its mission 
objectives. On the other hand, even though a proficiency evaluation program 
measures only roughly or crudely the factors relevant to mission objectives, 
it may provide a more adequate basis for the development of selection and 
Classification procedures than one which measures irrelevant factors with 
great accuracy. This phenomenon is generally not well recognized. A technical 
and detailed treatment of the issues involved is given by Thorndike (1949). 


Personnel Flow through the Man-Machine System 


A major characteristic of the personnel system which is essential for a 
dynamic personnel selection and classification model relates to the flow of 
Personnel through the system. We have previously pointed out that although 
New units of both men and machines are introduced into the system and 
existing units leave the system, the nature of the inflow and outflow for each 
I5 essentially different. The phenomenon of personnel flow into and through 
a system is utilized in developing essential aspects of the personnel selection 
and classification model. Personnel flow may be discussed in terms of loss of 
Personnel from the entire system or subsystems within the system, and also 
in terms of the induction of personnel into the system. 

The loss of personnel from subsystems within the system which does not 
Tesult from over-all loss is due chiefly to promotions and transfers. These two 
types of losses from subsystems are important because they imply different 
Problems in the logic of personnel selection and classification. 

_ Promotion may be of two kinds. The promotion may be along the super- 
Visory scale, where a person goes from one supervisory level to a higher one 
9r from a nonsupervisory level to a supervisory level. There may also be 
Promotions within a given kind of activity, where the change in performance 
's from a less difficult to a more difficult level of the same kind of function. In 
general, promotions can take place only if more room 1s being made at the 
‘op, because of expansion of the man-machine system, or because of the 
escape of personnel from the higher echelons of the total organization. We 

ave indicated that when a system is initiated it must rely in part on rational 

ases for its selection procedures. In general, the lower the validity of these 
ational considerations, the higher can we expect the ejection rate of personnel 

Tom the system to bs. In the case of transfers, We have the implication that 
Sither the design of the man-machine system is changing or that the classifi- 
Cation or assignment part of the selection procedure was not functioning 
adequately in the first place. The rate of transfer from one kind of job to 
mother at the same level of job requirements may be used inversely as an 
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may be ejected from it because they do not meet the requirements of available 
positions within the system or because the available positions do not satisfy 
them. Voluntary resignations occur because the individuals can presumably 
find more satisfactory jobs in other systems. But resignations may be due 
either to faulty classification or to inadequate functioning of persons in super- 
visory levels. In any case, promotion of any kind, whether from within a 
given type of activity or from one type to another (such as to a supervisory 
level), transfers, resignations, and discharges all imply vacancies to be filled 
by new personnel. 

It is the phenomenon of vacancies existing within a system which makes 
possible the development of a selection and classification model incorporating 
within itself a self-correcting submechanism. If by some omniscient scheme 
or program it were possible to staff a newly established man-machine system 
so that each person were perfectly fitted for the job he is to do, then the 
personnel problem would already have been solved. Realistically, of course, 
this is not the situation, and the most conspicuous features of the personnel 
selection and classification model are derived from the assumption that the 
hypotheses from which the model is developed are assumed to be subject to 
error, so that the model must provide for feedback and self-correcting mecha- 
nisms operating dynamically throughout time. 


Recruitment into the System. Both practically and theoretically, the 
recruitment problem must be concerned with the sources of personnel to be 
inducted into the system and the total supply available. The ratio of supply 
and demand is one of the constraints of the selection and classification model. 
The sources of personnel are important characteristics of a model because of 
their implications for both competing and cooperating man-machine systems 
which must compete with one another for the total unutilized reserve pool of 
manpower. These are problems we cannot discuss at great length here; their 
analysis seems to be far from complete at the present time. It is interesting 
to point out that during World War II recruitment procedures and recruit- 
ment sources for the armed services involved important theoretical assump- 
tions. Some of these assumptions were that the mission of the U. S. Army Air 
Forces required a higher order of abilities than did those of the other Army 
organizations, and that the job of pilot required a higher order of ability than 
did those of other types of aircrew personnel. The efficient operation of an 
ideal personnel selection and classification model is dependent to a certain 
extent on how sources of supply are defined and what restrictions are placed 
on these. 


Training Programs 


Cne important aspect of the functioning of a personnel system within the 
total system concerns those operations responsible for systematically modify- 
ing the performance characteristics of men within the system. The extent to 
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Which effective training programs, whether systematic or informal, exist within 
the system has considerable bearing on the development of the selection and 
Classification model. Other chapters in this book are concerned with proce- 
dures, devices, and concepts involved in modifying or improving the func- 
tioning of men within the man-machine system (compare Chapters 9, 10, and 
11). Here we shall be concerned only with those characteristics of the training 
Problem which have theoretical implications for the development of the 
Selection and classification model. This model must take into account how 
training relates to the mission objectives of the system, as well as how the 
training activities are distributed over the available jobs in the system. 

The relation of training programs to mission objectives may be discussed 
from the point of view of those programs concerned directly with these objec- 
tives, and of those programs concerned with preparing men to participate in 
these Objectives. These two kinds of training may readily be recognized as 
on-the-job training and formal, or classroom, training. In on-the-job training, 
Systematic supervision and tuition is provided the individual performing the 
functions of a particular job. On the other hand, many organizations have 
Vestibule schools or formal classroom sessions, where employees listen to 
lectures, observe demonstrations, read books, watch movies, operate simu- 
lators of actual hardware, and the like. The distinguishing characteristic of 
this kind of activity is that it is not in itself productive, as is on-the-job train- 
INg. There are, of course, many systems whose primary mission is that of 
formal training or the modification of human functions for the purpose of 
Satisfying mission objectives in other systems. Schools, colleges, and technical 
training schools in and out of the Department of Defense are such systems. 

Before hypothesizing the types of stimulus situation ‘aire = stapes 
9 evoke samples of relevant behavior, it is important to know for w fat in 
of training situations, if any, personnel are being selected. It is likewise 
important to know to what extent the ability to profit from experience Bu the 
Job unge; appropriate supervision and tutelage is required: or to what extent 
ability to profit from formalized classroom situations 1$ necessary. 

Within a given system, there may be a great deal of vriafon fom One 
type of job to another with respect to the extent and kinds of training pe 
cedures employed. Before it is possible to develop an over-all arii = 
c assification model for a particular man-machine system, ean s 
9 be provided as to the training available for each of the Le es a 
SYStem, These specifications must be set forth in sufficient detai aot ai the 
Worker characteristics may bë reasonably hypothesized, as outlined in 
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THE SELECTION AND CLASSIFICATION MODEL 


The terms “selection” and “classification” used in combination are some- 
what misleading. The implication seems to be that if one has a man-machine 
system which requires, let us say, a thousand men, one segregates or isolates 
from an available pool one thousand men for the system. After one has 
selected or segregated this group, he then classifies its members according to 
the various jobs within the system. This procedure leaves us in the dark as to 
how the men were selected in the first place. As a matter of fact, one cannot 
logically segregate or select the total number of men required for a man- 
machine system without selecting each man for a specific assignment in the 
system. 

But before the model can be developed, we must introduce certain basic 
concepts. We assume that certain hypotheses or sets of hypotheses 
oped from whatever source, and that then experimental data are 


and analyzed to test the adequacy of the hypotheses. The most con 
if not the most important, part of the model is 


rather than that of constructing them. Perha 
easier to specify the “hypothesis-verifying” 
the “hypothesis-construction” part, 

The basic concepts we require for the development of the model are 
three. They are attributes, entities, and measures. We assume that in the per- 
sonnel system, as in any system subject to scientific investigation, there are 
attributes or variables of some sort with which we are concerned. We also 
assume that these attributes do not exist in and of themselves but that they 
are manifested in some kinds of entities. In the case of the personnel selec- 
tion model, as in most psychological studies, the entities are human beings. 
We therefore have attributes or characteristics of human beings. The third 
basic concept with which we are concerned is that of measures or indices. 
We assume that, if entities are characterized by attributes, these attributes 


exist in some quantity, so that we have a measure for an attribute of an 
entity. 
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The Attributes 


Attributes may be defined as broadly or narrowly as we wish for our 
purposes. The personnel selection model which we shall develop involves two 
distinct types of attributes. One of these we shall designate as predictor 
attributes and the other as criterion attributes. Sometimes 
Settings, predictor attributes are called independent varia 
attributes are called dependent variables. In general, pred 
so called because they are used to make predictions. Crit 
those which are predicted or estimated by the predictor attr: 
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Predictor Attributes. First, let us consider some of the characteristics 
of predictor attributes. Because of the use to which predictor attributes are 
Put, they have certain essential characteristics. In the first place, predictor 
attributes are available prior to criterion attributes. Usually they are also 
much easier to evaluate than criterion attributes. For example, a test score 
is usually considered as a predictor measure. It is relatively inexpensive and 
takes perhaps not over a matter of minutes or hours to get a test score for 
Particular individuals. In the second place, predictor measures are usually 
Not regarded as important in themselves. The product of a response obtained 
from a predictor measure cannot be utilized in any useful sense. One cannot 
do anything useful with a person's response to a two-hand coordination instru- 
ment or to items in a test of mechanical aptitude. It is only because these are 
thought to be correlated with useful responses that they are of interest. 

Many kinds of predictor measures are available for the selection model. 
For example, one could consider the types of stimulus (input) situations 
described in Chapter 2 and the kinds of functions available for each of these 
Situations. Once could then introduce any number of variations or special 
Cases for cach of the stimulus situations and various manifestations of each 
type of response. For various combinations of these, one could then invent 
test situations or instruments for prediction purposes. Actually, many of the 
available test instruments may obviously be regarded as special cases. of 
Sensing, detecting, identifying, interpreting, and inventing with respect to stim- 
ulus differences, stimulus objects, OT symbols. Systematic efforts at such 
Classification have not been made. It is possible that such efforts would be 
Productive, not only in providing clues for better selection of predictor md 
aits, but also in providing clues for elaborating the system of human func- 
lons . 

icem pem think of predictor variables or attributes. as 
those attributes of individuals evaluated by means of tests and other selection 
‘struments prior to selection and classification for man-machine systems. 


iterion attributes are attributes which are 
lection and classification setting, the cri- 
dividuals exhibited in their perform- 


Criterion Attributes. The cr 
to be predicted. In the personnel se 


terion attri i of in 
attributes are those attributes dn à ms 
ance on the job. By definition therefore, the criterion attributes have certain 


important characteristics. First, unlike the predictor Vie they aie 
available until after the individual begins to function within t * — E 
System. Also unlike the predictor attributes, the cepe o n iw a x 
Utes is relatively expensive and time-consumine. or PETS Sant 
More expensive and time-consuming tO find out how ga ap 1 "» m can 
Y a B-52 bomber aircraft under all sorts of conditions than it is to fin out 


: hension test or on a simple 
nd i a mechanical compre t l 
d coordination tet. / rast to the predictor variable, the 


Cye-h ere in cont 

and co test. Also in © s cin ! 

Criterion rre portant and significant of itself. It is of direct concern 
able is im ane 


252 Personnel Selection and Classification 


to mission objectives whether a pilot can fly a B-52 effectively; or whether a 
technical sergeant can properly carry out his specific functions in the launch- 
ing of a ballistic missile. . 
We have previously discussed the problem of proficiency evaluation as it 
impinges on the selection model. Ideally, we should have a common language 
which will bridge the gap between criterion measures and predictor measures; 
that is, we should be able to evaluate criterion or job performances in terms 
of the basic types of response functions in relation to specified types of 
stimuli. Chapter 12 is primarily concerned with such problems, and no 
attempt will be made here to characterize the different types. of criteria. 


Dual Functions of Attributes. There is a third kind of attribute, which 
may serve a dual function. At one period in time the attribute or variable 
may be a criterion attribute and at a later time it may be a predictor attribute. 
For example, performance on a particular job may be a criterion attribute at 
one time, but this performance may be also used to predict performance on 
a job at a later time. Performance on a lower-level job may be regarded as a 
criterion attribute which has been Previously predicted by some predictor 
variable, but which may also be used as a predictor to predict success in a 
higher-level job. 

One example of this is found in systems which have established training 
programs as part of the man-machine system. Before the individual is ad- 
mitted to the training program, whether on-the-job or classroom, one may 
predict success in these training activities, Performance in the training activi- 
ties would then constitute the criterion attribute. But the same performance 
may also be used to predict success in later phases of productive activity on 
the job; in this case they are regarded as predictor variables or attributes, 


The Entities of the Selection Model 


We have said that the attributes must inhere in some kind of entities, 
and have pointed out that in the personnel situation the entities are individual 
persons. In order further to develop the Personnel selection model, 
examine the concept of entities more specifically. It is import 
the entities of the selection model to consider two m 
we shall call the experimental entities, because it is o 
the selection model is developed. The other we shall 
ties, because these are the entities to which the 
experimental entities are applied. 
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We begin to see now why the concept of flow of personnel in the man- 
machine system is important for the selection model. The model assumes that 
we have a group of experimental entities on the basis of which selection pro- 
cedures can be established, and that these Procedures may then be applied to 
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entities known as the administrative entities. Specifically, we find out how to 
make predictions on the group of experimental entities and we make these 
predictions for the administrative entities. 

We have said the model assumes that for an attribute we have a measure 
With respect to an entity. These measures must be numerical if the model is 
to have more than journalistic significance. 

Let us now summarize these basic concepts in terms of Figure 7.1. In 
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Measures of these functions for a group of individuals. A particular type of 
Nalysis known as factor analysis has as its basic function the determination 
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of the number of statistically or functionally independent attributes within a 
specified system. Although authorities disagree, there is good evidence to 
indicate that the total number of socially significant kinds of different be- 
haviors which can be measured by objective procedures does not exceed 
thirty or forty. This applies to psychological functions rather than to specific 
achievement variables involving specialized knowledge or facts. 

Furthermore, the model assumes measures of the experimental entities 
on the criterion attributes. Very rarely, however, is it possible in practical 
situations to have measures of experimental entities on all the criterion attri- 
butes within the system. In a later part of this section we shall consider in 
more detail the submatrix of measures in Quadrant II of Figure 7.1 with 
respect to the pattern of missing data. 

Next let us examine the essential characteristics of the administrative 
entities represented by the lower part of the diagram in Figure 7.1. An 
essential characteristic of the administrative entities is that their number or 
quantity may not be specifically defined. This is indicated by the fact that we 
start with the numbers 1, 2, 3, but then do not go on to some general symbol 
for the total number, and that we have a broken line at the bottom. The 
administrative entities, let us remember, are those to which the procedures 
derived from the experimental entities are applied. We may have a fairly 
unitary group of entities, such as an Army quota calling for a thousand 
inductees at a time and place. But more typically, in most man-machine sys- 
tems the experimental entities come in a more or 


; less even flow from day to 
day, as is the case with job applicants for positions vacated as a result of 
expansion of the system or exits from the system. The fact that Quadrant III 


is shaded means that we have measures on all of the experimental entities 
for the predictor attributes. The fact that the space in Quadrant IV is blank 
indicates that we do not have measures on any of the experimental entities 
for any of the criterion attributes. Figure 7.1 is the fundamental model from 
which we must work in developing the elaborated model of the selection and 
classification subsystems. The basic scheme of the selection model shown in 
this figure is a special case of what is known as the generalized multivariate 
prediction model. 

We have said that the selection model implies some sort of index or 
measure of an attribute for an entity. In Figure 7.1, the entries in Quadrants I, 


II, and III are figures or indices of this kind. We shall now discuss in more 
detail the measures of the attributes. 


The Measures of the Attributes 


We need in our model two basic concepts in discussing measures of the 
attributes for the entities. First, as we have indicated already, these measures 
must be in the form of number symbols, or such that they can be translated 
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to number symbols. Second, we need the concept of more than one estimate 
of the attribute for the entities. That is, we need the notion of replication or 
repeated samples of behavior with reference to a particular defined function 
or performance. 


Numerical Score. Although in many cases it is not feasible to have a 
Measure or an index of performance for each entity with respect to each 
attribute, it is essential that, if such an index is available, it be in the form of 
à number. These numbers often are scores on à test, such as the number of 
items correctly answered according to a predetermined scoring key. Or they 
can be the number of seconds or minutes it takes to assemble a particular 
group of objects, the number of errors made in typing a piece of copy, the 
Tating assigned to a person by a supervisor, or the monthly salary of a person 
1n a particular job. 

As a special case, the number may be restricted to either zero or one. 
This is called a dichotomous measure, and the fact that we may have such a 
simple type of measure makes it possible to quantify practically any attribute 
in which we may be interested. For example, sex may be regarded as an 
attribute, Here we may assign a value of one for female and zero for male. 
Another variable of this type of attribute is college graduation. We may give 
à score of one for "graduated from college" and zero for "did not graduate." 
Many important models of scientific research and investigation in the social, 
biological, and even physical sciences utilize in their designs this notion of 


Zero-one measures. 


. Replication. The second characteristic of measures which is important 
i the selection and classification model is that of replication. By replication 
IS meant that there is more than one measure of an attribute for an entity. 
simple example is a test score. Objective test scores are made up of scores 
On individual items in the test. A mathematics test includes a number of 
Items, and the score is a function of the total number of items answered cor- 
rectly, This is a very simple example of replication, or of taking a sample of a 
Number of different responses from the individual with respect to the same 
ind of stimulus situation. We may also consider repeated responses to the 
Same stimulus pattern or group of patterns in successive time intervals. For 
example, we may give the same test to a person today and again next week. 
ere we have two different measures for the same individual or two different 
Measures of the same test. Or in the case of an observer's rating, the observer 
i weeks or months. We may also 


may gi i sive 

give repeated evaluations for succes í s 
have Observations or evaluations with respect to the same function by dif- 
fen lication 1$ crucial in the multivariate 
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Selection and classification programs may fall short of being maximally 
effective because they do not take into account certain characteristics which 
the measures of attributes should satisfy. All of these characteristics of meas- 
ures must be considered in relation to the particular group of individuals to 
whom the model is applied. 

Measures of attributes must satisfy certain conditions with respect to a 
particular experimental group. The first of these we shall discuss is that of 
average level of performance. The test of a function should not be too easy 
or too difficult for the group of individuals who are supposed to perform the 
function on the job. For example, an arithmetic test used to predict success 
on a job should not be so difficult that people functioning satisfactorily on 
the job make poor scores on the test. Neither should it be so easy that all 
persons could finish the test with a perfect score in the time allowed. 

Another characteristic of the measure is that when applied to the group 
which is to be considered for jobs there should be variation in 
those tested or measured. Some people should make good test 
poor, and others in between. This means that the standard deviation of the 
distribution of scores should be adequate to make possible differentiation 
among group members. Even in the case of dichotomous measures, the experi- 
mental group must have some people who get one scores and some who get 
zero scores. Otherwise the attributes will not function in the selection and 
classification model. For example, sex cannot be used as a potential predictor 
variable in the experimental group unless some of the persons are males and 
some females. A measure of an attribute which does not discriminate among 
the members of the experimental group is of no value in the selection and 
classification model. Other things being equal, it may be said that those 
attributes which have the largest dispersion in the experimental group will 
serve the most useful function in the selection and classification procedures. 

A third characteristic of the measures applies more specifically to con- 
tinuous measures than to dichotomous measures. This characteristic is known 
as homogeneity. It applies to those measures which may be regarded as made 
up of replications. For measures of this kind, it is desirable that the stimulus 
elements be as nearly comparable as possible. That is, the items within a test 
should all be of the same type, in the sense that they require the same sort 
of response function. This point can be put in Statistical terms. If we have 
Separate measures on the individual parts of the sample of behavior for each 
of a group of individuals, the intercorrelations of these measures of unit 
responses should be as high as possible. In particular, the measure of a unit 
response to an objective test will ordinarily be zero or one. The responses 
to unit elements of Psychomotor or eye-hand coordination tests may be arbi- 


trarily defined units of time, such as the first five seconds of performance oF 
the next five seconds, 


scores among 
scores, others 
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A fourth important characteristic of the measures is that they should be 
objective. The method of arriving at the numerical index should be such that 
there is high agreement among observers as to a particular person’s measure 
on a particular attribute. Psychological tests are good examples of objective 
Measures. In general, it is not a matter of argument whether the individual 
has 20 or 30 items correct according to a specified answer key. On the other 
hand, if the task is to write an English essay on a given subject, then there 
may be considerable disagreement as to whether a person should get a score 
of 75 or 100. 

Another characteristic of measures which applies both to criterion and 
Predictor attributes is that of stability. This refers to the consistency of the 
Measure for a particular individual over a period of time. If the criterion func- 
tion to be measured is such that the measure of it fluctuates wildly up and 
it may be impossible to predict. It is important 
ance which will exhibit stability for individ- 
his characteristic are said 


down over a period of time, 
to construct measures of perform 
uals over a period of time. Measures which exhibit t 
to be reliable, and those which do not, unreliable. . 
Reliability is of particular importance for measures of predictor attri- 
butes, If the measure of a particular function today gives little indication of 
What the measure of this function will be next week or within a month, we 
cannot expect such a measure to predict later performance on the job. It 
Should be noted, however, that stability may be achieved simply by taking 
evaluations of performance over a sufficiently long period of time. It should 
be noted that stability, like homogeneity, involves the concept of replicated 
evaluations of the same attribute for a given entity. The essential difference 
€ ween the notions of homogeneity and stability is that in homogeneity we 
are concerned with responses to à number of the same type of stimulus pat- 
terns, whereas in the case of stability we are concerned with repeated re- 
SPonses to the same stimulus pattern over a period of time. In both cases, we 
Collect a number of behavior samples involving the same function. . 
_ The next characteristic of the attributes we shall discuss is called speci- 
ficity. The first five characteristics we have mentioned—level, discrimination, 
Omogeneity, objectivity, and stability—all refer to a measure of a specific 
attribute, Specificity concerns the relationship of one attribute to another. It 
desirable in the personnel classification and selection model that the pre- 


ictor attri z ä f one another as possible. In statistical 
ependent O ei E 
utfibites hë 48 TEP ng the predictor attributes should be as 


ermino] i i 
o relations amo E A : 
OW as e i ctt for this is that, the higher the intercorrelations 
tween two vari bles the more nearly do they mesure | Ley inmane" 
ariables, r measuring them. A commonly 


ven f 
thougl different methods fo : 
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“ational analysis on which the measures are based, while appearing to yield 
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logically different psychological variables, nevertheless fails to Lia € 
tially different functional unities. Another reason may be that, al ti ough cue 
tions have been described which are in themselves relatively indepen en 
and unitary, efforts to measure these functions do not result in instruments 
which actually do measure different functions. Some of the instruments may 
duplicate others even though they may seem to be different. . 

It should be obvious that merely by applying different labels to different 
instruments we do not ensure that we are measuring really different kinds of 
functions. Nevertheless, this assumption is made in many presumably respon- 
sible testing programs and frequently leads to disappointing and misleading 
results. The higher the intercorrelations among a set of predictor measures, 
the less useful will these measures be for predicting in which of a number of 
different kinds of criterion activities (job functions) a person will be most 
successful. Another way of stating this important principle is that the higher 
the intercorrelations among a set of predictor measures, the less useful these 
will be for personnel classification. 

The final characteristic of measures which we shall discuss is what has 
traditionally been called validity. In recent years, a number of different kinds 
of validity have been defined, but only one of these will concern us in the 
personnel selection and classification model. B 
mean the accuracy with which it 
More generally, 


y the validity of a measure we 
predicts performance on a criterion activity. 
statistical validity refers to the correlation between a predictor 
and a criterion measure. We refer to the coefficient of correlation between a 


predictor measure and a criterion measure as a validity coefficient. In this 
sense it is clear that a test may have, not a single validity, but as many 
validities as there are criterion measures with which it may be correlated. 

It is also possible to speak of the validity of a criterion measure, 
although this is not as commonly done. For example, we may consider super- 
visors’ ratings of performance on a particular job as a criterion measure. We 
may then raise the question, “Do these ratings give an indication of how well 
the person is performing in relation to the specific mission objectives of the 
particular job, or do they instead reflect personal interactions between the 
supervisor and the employee?" In other words, do these ratings also indicate 
how well the supervisor likes the employee? If this latter is the case, and he 
likes the employee whether or not he is adequately performing the specific 
mission objectives of the job, then we may say that the criterion measures 
àre not valid measures of job performance. Another example may arise when 


objective achievement or information tests of some sort are used as criterion 
measures for individuals in a particular job. These measures are objective; 
they may also be stable and ri 


eliable. They may place emphasis on what the 
subject knows about a job, such as technical terminology, but they may not 
indicate how well he is actually performing the specific mission objectives of 
the job. In this case, while the test scores may be regarded as objective and 
reliable measures of the criterion performance, they may not be valid 
measures, 
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It is often difficult to separate these characteristics of a measure in the 
case of criterion activity. In many man-machine systems, it is particularly 
difficult in team situations to determine for any particular individual just how 
well he is performing the mission objectives of the job. In a straightforward 
assembly job, where each person works independently, valid proficiency 
measures may be more easily arrived at. In general, it is probable that insuf- 
ficient effort has been devoted to the designing of man-machine systems so 
that the contribution of a single individual can be isolated or segregated from 
the output of the group to which he contributes. Certainly in an assembly 
line the productivity of individuals is largely paced by the tempo of the assem- 
bly line itself. 

From the motivational point of view, it is possible that systems designed 
SO as to permit each individual to contribute independently as much as he can 
may be generally more productive. This is a separate problem, however, and 
is more appropriately expanded in other parts of this bo . The 
relevance of this point for personnel selection and classification is that the 
less the productivity of a man-machine system depends on the independent 
Sums of the productivity of the individuals in the man subsystem of it, the less 
effective can be the development and validation of a selection and classifica- 
tion model. ; T" 

In terms of Figure 7.1, then, we may summarize the characteristics [9 
Measures as follows. The characteristics of level, discrimination, homogeneity, 
objectivity, and stability are concerned primarily with the measure of each 
Of the individual attributes in Quadrant I. With the exception of homogeneity, 
these also apply to the measures of each of the attributes in Quadrant II. 

he characteristic of specificity refers to the interrelationships of the attri- 
butes among themselves and involves only Quadrant 1. at the minia 
activities are based directly on arbitrary job designations, there is ne aum 
Can do about the interrelationships of these, so that ordinarily specificity = 
Not involve Quadrant II.) Finally, the characteristic of validity eri the 
Mterrelations or correlations of the attributes in Quadrant I with those in 


"adrant II, 
We have referred several times to 


and classificati ires not on 
cation model requires " F 
also the verification of these hypotheses. This requirement of the model is 


dictor and cri- 
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ected in Figure 7.1 by the fact audent dms 
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Setting up a man-machine system “from scratch” t sn a both predict 
Broup of entities in the man-machine system on whom we have I" or 
not existed previously, therefore the 


ed the tasks for which criterion 


ok. The essential 


the fact that the complete selection 
ly the development of hypotheses but 


ex 
Perimental group could not have perform 
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For any such newly constituted man-machine system, some assumptions 
must be made about the variables important for predicting success in the 
newly constituted system and how they should be measured. It is possible to 
develop hypotheses structured somewhat along the lines of Chapter 2 on the 
basis of information provided by the kinds of analyses outlined in Chapter 6. 
Hypotheses must be formulated about the extent to which each of these 
predictor attributes enters into predicting success in each of the jobs for 
which personnel are to be selected and classified. In other words, there must 
be some procedure for weighting test scores differentially for each of these 
particular jobs, so as to get predictions of success for each job. 
however, one must operate on the basis of hypotheses alone. Only later, as 
men are absorbed into the system and measures of criterion performance 


become available, can these hypotheses be tested and modified on the basis 
of experience. 


At this point, 


Prevalidation Administration 


Both from the theoretical and 
that whatever measures we ado 


bles satisfy the required char- 
ustrated by the simple diagram 


3 e X similar to that in Quadrant I of Figure 
7.1. Operationally, this means that, having worked out a set of predictor 


measures, we administer these to a prevalidation group, so that our hypotheses 
with reference to the essential characteristics of the measures may be tested 
and so that the measuring instruments may be modified on the basis of these 


Predictor attributes 


Prevalidation entities 


N 


Figure 7.2. The matrix of n predictor measures on a group of N 
prevalidation entities. 
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results. Frequently, selection instruments turn out to be much less effective 
than they might be because this preliminary feature of the general model has 
been neglected. In general, the kinds of analyses to be performed on the pre- 
validation entities are those of difficulty, dispersion, homogeneity, stability, 
and specificity. These are discussed in detail in textbooks on measurement and 
evaluation, such as Cronbach (1960) and Anastasi (1954). 

It is of some relevance to the development of the model to consider the 
Source of the prevalidation entities. We have several possibilities. First, these 
entities may be the same as the original group of men taken into the new 
System. In general, however, this is an extravagant and impractical procedure 
because, in any case, the revised set of predictor measures must be applied 
to the group considered for induction to the system. If the prevalidation mate- 
rial is given to this group, the group will then have to be subjected to the 
second administration after the material has been revised to meet the required 
Specifications. 

It is usually desirable to ad 
the system which may be available for 
Practice it may be difficult to find suc 
inductees are often used for such purposes. 
times possible to get school systems to coop cts. If thi 
system is an autonomous subsystem of a larger system, such as a missile divi- 
Slon of a large aircraft company, then it may be possible to use the time of 
*mployees in other parts of the company for such experimental purposes. 

It is not uncommon in the development of predictor measures for a 
Selection and classification procedure of a new man-machine system to use 

atteries of material already available. This is common in the armed services, 
Where extensive work has been done on the development of personnel selec- 
tion and classification batteries. Unfortunately, however, many ar these bats 
teries do not exhibit adequately the essential prevalidation characteristics of 


a ; 
battery of predictor measures. 


minister the material to groups external to 
such experimental purposes. In actual 
h groups. In military situations, new 
In industrial situations it is some- 
erate in such projects. If the 


T " 
he Criterion Data 


The selection and classification model cannot be completed until we have 
Measures of criterion performance on the men operating wine the new sys- 
tem. Let us now consider in more detail the nature of the ortenon SUAT: 
It may first be observed that, typically, in the n and Vk ipe 
rel the chief emphasis in the past has been on "Y ection phage ie B a 
icular job has first been thought of and has le p^ ae ^ ea E ing 
.Düttery of tests or measures to predict success in this Job. e battery is 
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itself only with a single column of the criterion attributes in Figure 7.1. It 
should be observed, however, that what we have indicated for convenience 
as a single column may actually involve a number of different kinds of func- 
tions. If the attribute is on the basis of job designation—such as typist, wed 
operator, or pilot—the job itself may be complex and require a variety o 
different functions. This, then, suggests the possibility of a criterion of over-all 
effectiveness as a pilot or a typist or a lathe operator. But there is also the 
possibility of breaking the job down into a number of different functions, such 
that performance on each of these may be evaluated. The kind of analysis 
which may be undertaken has been described in Chapter 6. 

For a single job, then, we may have just one measure or we may have a 
number of functions, in each of which an entity has an evaluation for a given 
criterion. The question of whether one wishes to predict performance in each 
of these attributes or to combine these evaluations into an over-all measure 
of proficiency is relevant for the model. The technicalities involved in carry- 
ing out these alternatives are discussed in detail by Thorndike (1949). 

The model we have just discussed is not useful for classification pur- 
poses, since it takes into account only a single job or kind of activity. The 
appropriate model from the systems point of view involves all the jobs within 
the system. This means a consideration of the entire submatrix represented 
by Quadrant II of Figure 7.1. Here we have a number of different criterion 


attributes or job designations. But again it can be seen that the alternatives 
exist of an over-all evaluation for each job designation or of breaking each 
job down into a number of different kinds 


of functions and providing meas- 
ures of proficiency on each of these. 


Theoretically, one may have the same s 


ystem of categories for segre- 
gating the functions required for each of the jobs. Ideally, from a psychologi- 
cal and perhaps even from a practical point of view, this is 


the best approach. 
The same categories would then occur in Figure 7.1 for th 


€ predictor and the 
criterion attributes. The criterion measures would then be set up on the basis 


of psychological or behavior functions rather than job designations. This 
procedure would have immense advantages relative to the problems involved 
in the fragmentary criterion-data matrix. 

In general, there are enormous problems in getting the actual evalua- 
tions on the job on the basis of a common set of designations and of making 
the measures comparable from one job to another and one supervisor to 
another. The practical difficulties, however, should not blind us to theoretical 
and practical advantages or prevent us from seeking methods of achieving 
such a treatment of criterion attributes. Perhaps one of the most pressing 
needs is not only the development of a common terminology for functioning 
of men and machines, but also the extension and utilization of this termi- 
nology in the development of predictor and criterion measures. We would then 
predict performance in terms of these functions. 
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Criterion Measures. We are now ready to discuss characteristics of the 
pattern of measures in the typical criterion submatrix. We have pointed out 
that, typically, job designations are used for criterion attributes. If this is the 
case and if we are to have criterion measures on all entities for all attributes, 
this means that each person in the experimental group must have an oppor- 
tunity to perform on each of the jobs within the system in order to determine 
his Proficiency in each of these. This is precisely the model which has been 
Suggested by some experimenters in the field. 

Specifically, the proposal is made in connection with technical training 
Schools where courses in a large number of technical specialties are provided. 
A battery of differential prediction tests is given to students before they enter 
the school. Each student is then required to take all the courses, and each 
Course or technical specialty is then regarded as a criterion attribute. Here, 
then, we have measures for each of the students on each of the criterion 
attributes and also measures for each of the students on each of the predictor 
attributes, This is the ideal model from a theoretical point of view, but in 
Beneral it is not feasible in operating man-machine systems. However, the 
evaluation of job performance on the basis of a standard or universal set of 
behavior categories would best exemplify this approach to a measure on each 
Criterion attribute for each experimental entity. — ' SA . 

A second type of criterion submatrix is one Im which the criterion attri- 

Ute is defined in such a way, and the administrative structure and procedures 
Of the system are such, that for any given entity there may be measures of 
Performance on several, but not all, of the criterion attributes. Practically, 
What this means is that within a system, persons may participate in several 
different jobs. Their time may be divided among two or more jobs or they 
May be transferred over a period of time from one job to another. The most 
common example of this ii of criterion participation is in an 
Educational system such as a college or university, where the student may be 
regarded as the man subsystem of the system and the output is performance 
On examination papers, lab work, and the like. The measures of these per- 
Ormances. then, would be grades assigned to the student in the various course 
areas. In such à case, a student would ordinarily have grades in a number of 
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Figure 7.3. 


The typical case of m criterion measures and N experi- 


mental entities, where not all the measures are available for all the entities, 
and where the missing data exhibit no systematic pattern, 


single row of the submatrix, An example of this type is indicated schematically 
in Figure 7.4. This type of criterion data submatrix is by far the most common 
in man-machine systems where a criterion attribute is defined in terms of a 


job designation. This we may call the mutuall 


submatrix. 


y exclusive type of criterion 


It may be seen, therefore, that methods of describing performance within 
the man-machine system and the structure of the system itself determine very 
materially the Kind of criterion submatrix which becomes available for the 


experimental group of entities. 


Criterion attributes 


Experimental entities 


Ln | 
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In the model represented by Figure 7.1 there are straightforward solu- 
tions for the statistical establishment of prediction procedures for personnel 
selection and classification. However, the situation in most man-machine sys- 
tems is not of this type, and even though there is a possibility of defining 
criterion attributes to yield such a matrix, the problems involved are far from 
Solved. In actual practice, we are almost entirely restricted to the types shown 
in Figures 7.3 and 7.4. 

The mathematical procedures for using the data from the predictor and 
criterion attributes on the experimental entities as a basis for developing pre- 
diction procedures for the administrative entities are well known for the case 
in which the criterion submatrix has no missing elements. These are known 
as multiple regression techniques (Thorndike, 1949). Special problems, how- 
ever, are encountered with the more typical cases represented by Figures 7.3 
and 7.4, where we have missing data in the criterion submatrix. These we shall 


discuss briefly. 


Criterion-Data Analysis. The methods for handling the more typical 
Cases indicated in Figures 7.3 and 7.4 have not been nearly as well worked out 
mathematically as those for the more atypical case of a complete criterion-data 
use is made of subgroups of experimental entities, 
Including in each analysis only those cases which have criterion estimates for 
4 particular criterion attribute. Such a procedure is not optimal for the case 
Of Figure 7.3, where there are indications about the interrelationships of per- 
formance on various criterion attributes only to the extent that there is over- 
lapping among those who have performance estimates on two or more of the 
Criterion entities, However, for the case of Figure 74, where mutually exclu- 
sive Participation rather than discretionary participation occurs, ewe pro- 
cedures for estimating criterion performance on the various criteria from the 
Predictor data utilize all the information available. Here, as in the over- 
lapping case, the analysis proceeds by segregating subgroups according to 
those having criterion data for a single criterion category. Such subgroups are 


9| ! 
f course mutually exclusive. 


Submatrix, In general, 


Sample Selection 

casures of both criterion and predictor 
for a particular group. But varia- 
ction, not only of the way the 
lar sample on which the meas- 


. We have said earlier that the meas 
qributes must show dispersion or variation 
ion with reference to an attribute is a fun 


attribute į f the particu. 
€ is measured, but also o! p l he 
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e functions measured by the assessment procedures, the pads inim 
themselyes can show no differences. For example, one cannot show differences 
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"Bh. As an instance ofa special case, one may have a set of valid predictor 
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measures on the basis of which persons are selected for a particular em 
Those who fall below a critical score are rejected. For those who enter tl e 
system, criterion measures subsequently obtained will not have as much varia- 
tion as if the entire group had been accepted. Selection procedures which 
affect directly the dispersions of some of the predictor attributes may also 
affect the degree of relationships among the various predictor and criterion 
attributes. A body of theory and technique is available for handling such cases 
of sample selection (Meredith, 1958). : 

These techniques apply particularly where the selection is of a systematic 
sort. One example is when all persons exceeding certain critical scores are 
assigned to one kind of activity—say, pilot training—and others with other 


scores to bombardiering or navigation. Methods of analysis are concerned 
with the effect of such selections directly on the means, 


relations of the attributes on which direct selection is mad 
variances, and covariances of other variables affected by 
Some of these techniques may be applied with limite 
cases of criterion subgroups such as indicated by the 
Here, however, we may have indirect, rather than direct, 
ple, students decide which courses they are going to take, and we do not know 
in general on what basis these decisions are made. Also, persons decide to go 
into a particular job, instead of being assigned to it on the basis of certain 
predictor measures. The problems of adjusting data and incorporating the 
adjustments into the selection and classification model are important. The 
techniques currently available still leave a number of problems unsolved and 
require a number of assumptions which are known to be imperfectly satisfied. 


variances, and cor- 
e and on the means, 
direct selection. 

d assumptions to the 
Figures 7.3 and 7.4. 
selection. For exam- 


KINDS OF PREDICTION 


We have discussed various aspects of the prediction of criterion per- 
formance from predictor variables. Now we shall consider more systematically 
the kinds of predictions which are relevant to the personnel selection 
classification model. We shall discuss five different models in more or 
common use. These are the single-criterion model, 
prediction model, the multiple differential-prediction model, the optimal- 
classification model, and the multiple discriminant-function model. 


In the single-criterion model we have indicated th 
defined that there is onl 


distinctions among vario 


and 
less 
the multiple absolute- 


at the system is so 
y a single criterion attribute. This model makes no 

us kinds of activities carried on in the man-machine 
System. It assumes a single-criterion attribute, so that in Quadrant II of Fig- 
Uke: 7.1 there would be only one column. This assumes that the interest iS 
in predicting success only in a very general way within the man-machine 
system, irrespective of what functions within the system a man may perform. 
The screening procedure used in the aviation cadet program during World 
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War II is an example of this. An Aviation Cadet Qualifying Examination was 
given to persons in the ground forces who wished to go into air-crew training. 
A single score was provided by this test, although the test consisted of a num- 
ber of different parts which measured several different kinds of psychological 
functions. The assumption involved in this procedure was that if a person 
achieved a score above a specified cutting point, he would be successful in 
Some kind of air-crew activity. Another example of the single criterion is 
found in attempts to predict success in college irrespective of what courses a 
Student takes. The Army General Classification Test is another example of 
the single-criterion model. The prediction is made that a person will or will 
Dot be successful in the Army, regardless of what kind of assignment he is 
Biven. 

In general, the single-criterion model is concerned primarily with rejec- 
tion of unqualified individuals. Cutting scores are established, so that it is 
a reasonable assumption that all people below this cutting score will most 
likely be unsuccesful in almost any kind of activity within the system. People 
above this score may be successful in some kinds of activities and not in 
Others, The basic function of a screening test is to eliminate those persons 
Who will almost surely not succeed satisfactorily in any of the major activities 
of the man-machine system. Thus the single-criterion model may be con- 
Sidered a rejection, rather than a selection, model. . u 

The next model we shall consider is that of multiple absolute-prediction 
(Horst, 1955). Here one assumes a set of criterion attributes such as are 
indicated in Figure 7.1. The number of attributes depends on the system and 
the manner of definition of the criterion attributes. It is assumed that with an 
adequate set of multiple predictors and the use of appropriate ape E 
cedures for a group of experimental entities, estimates of success in each x a 
Number of criterion categories can be provided. The pus peri san 
Interest is centered on predicting absolute performance in each of these cri- 


teri : 

p of prediction has been called multiple difjerential-predic- 
tion (Horst, 1954). Here again success is predicted with prem E E 
number of different criterion activities such as those indicated in s ds 

©Wever, in this case the emphasis is not in predicting Desa oan 
19 much as in predicting relative performance for cach of the P aka a F 
Nes. This model assumes that each person can make ers n wor aang 
Contribution in any particular job activity, even though it may 5e very 1 


and that one is interested in knowing in which of the onesies Eun re 
Make the best contribution. The aim is to provide setti. Differential re- 
© Prediction of relative success in various job possi A ^» ecu mier 
lion can be important in the case of predat i a person will do abso- 
ithin wide limits the concern is not with how We M x a 

Utely in a particular course area, but rather with which course areas or 


ajors he js apt to do best in. 
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In most practical cases the differential-prediction model must hec 
plemented with the multiple absolute-prediction model, for the PIOS 
nearly always valid that in any kind of activity à person must have a mi 2 
amount of ability in order to continue in the activity. For example, it is usually 
not enough to know that a person could do better in English than in mathe- 
matics if we know that his absolute performance in either of those would be 
failing in any case. x 

One important distinction between the multiple differential-prediction 
model and the multiple absolute-prediction model is that for any given set of 
predictor measures the techniques and the formulas for making the predic- 
tions are exactly the same. However, the predictor variables most useful for 
multiple absolute prediction are generally not the same as those which are 
most useful for multiple differential prediction. For example, if there is a set 
of criterion measures defined, Say, according to job designation, such that 

there are high correlations among the various criterion measures, this means 
that these measures have one or more factors in common which cause the 
high correlations. If in the group of tests there occurs a subgroup which pre- 
dicts the factors common to all the criterion measures, this group will be 
useful in predicting absolute performance in each of the criterion measures. 
But if one is interested only in predicting differences in criterion measures, 
the tests will not be needed which measure whatever the criterion measures 
have in common. It is in this sense that multiple absolute prediction and 
multiple differential prediction are essentially different. It should be noted, 
however, that if that group of tests has been selected which gives the best 
multiple differential prediction and if certain tests are added to it which 
measure also the factors common to the criterion variables, the result will be 
a total battery maximally efficient for both differential and absolute prediction. 
It can be seen from the description of the multiple absolute and the 
multiple differential models that both of these offer possibilities of classifica- 
tion or assignment of men to the various jobs within the system. Suppose, 
however, that we have a group of predictors, as su 
paragraph, which are capable of both optimal abso 
diction. These predictions should then provide a 
which of the various job categories each of a gro 
assigned. Suppose that each person is assigned to t 
diction of success is highest. (The assumption is mai 
for the various predictions are available.) Suppose t 
of jobs to be filled and that 20 persons are requi 
further that 1000 persons are available to fill thes 
to be assigned the job for which his 
to expect that precisely 20 would b 
personnel selection and classific 
available sources the number of 
For these reasons, 


ggested in the preceding 
lute and differential pre- 
basis for determining in 
up of persons should be 
he job for which his pre- 
de that comparable scales 
here are 50 different kinds 
red for each job. Suppose 
e jobs. If each person were 
prediction is highest, there is little reason 
€ assigned to each type of job. The ideal 
ation model is one which will yicld from the 
persons required for cach job. 

the need is evident for a system for making estimates 
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of success which will take into account not only the abilities of the persons 
but also the number of persons required for each job. This model, called the 
optimal classification model, is much more complex mathematically than either 
of the other multiple-criterion models which we have considered. Various 
methods are, however, available for approximating a solution (Horst, 1960). 
The general principle is to assign each person so that the over-all effective- 
ness of the organization will be maximal. This means that people should be 
assigned so that the sums of the estimated performances for all persons with 
Tespect to the jobs to which they have been assigned are maximal. 

One elaboration of the optimal classification model provides not only 
for the classification of individuals within the system, but also for the rejec- 
tion of part of the applicant group from the system. This model assumes that 
the group of individuals available is larger than the total number required 
for the system. In addition to optimally classifying or assigning each of the 
individuals within the system to mect quota demands, it also rejects the 
Temainder of the group not required, in such a way that those rejected would 
have contributed least to the over-all efficiency or productivity of the system. 

A fourth kind of prediction model is based on what is called multiple 
discriminant-function analysis. This model also assumes a multiple-criterion 
system, as in the case of the other three mutliple-criterion models. It assumes 
further that each criterion category corresponds to a special class of indi- 
viduals and that a person belongs only to one of these classes. The technique 
calls for utilization of the predictor variables in such a way that individuals 
can be assigned to the classes to which they “really” belong. The general 
notion is to construct predictor measures in such a way that the number of 
misclassifications resulting from the use of these measures is minimal. , 

The philosophy underlying the multiple discriminant-function technique 
May be subject to some question. From what we know about human beings, 
they do not belong to only one of a specified number of classes. It is unlikely 
that psychological functions are of this all-or-none character; rather, an indi- 
Vidual tends to have more or less of some kind of proficiency. Various tech- 
niques have been worked out to combine multiple discriminant-function 
techniques with other multiple prediction procedures (Horst, 1956). 


Predictor Selection 


Perhaps the most important problem of all, and one which we have 
arely touched on, has to do with the verification of the personnel selection 
Model. It is coricerned with empirical methods for selecting predictor variables 
Which will be most efficient in the predictor system. Considerable work has 
been done on predictor selection techniques. Actually, however, all of these 


leave much to be desired, because of the fact that theoretically one may in- 


Clude an extremely large number of potential predictor variables in a multiple 


Predictor system in licu of a demonstrably satisfactory rational approach, All 
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of these methods encounter problems involving degrees of freedom. Methods 
have been worked out which attempt to tell when an adequate number of 
predictors have been selected for a particular system. These, however, appear 
to be far too conservative. But both the technical and philosophical problems 
involved in this subject are beyond the scope of this chapter. 


Predictor Formulas 


So far we have implied that there are specific procedures for utilizing 
the predictor measures in making predictions of the criterion measures. It is 
not possible in this chapter to go into the technical details and statistical 
procedures involved. However, a brief general discussion may be useful. 

In general, there are two ways that predictions of success can be made 
from the predictor variables. In either case, these are established on the 
experimental entities indicated in Figure 7.1. The first of these may be called 
the method of optimal cutting scores and the second the method of linear 
combination. 

The method of optimal cutting scores proceeds on the basis that for each 
criterion attribute there can be established a set of cutting scores on the 
predictor variables so that persons in the experimental group who meet these 
cutting score requirements also have sufficiently high criterion measures. 

The method of linear combination assumes that for any particular cri- 
terion measure a linear combination of the predictor measures can be found 
so that for the experimental cases on whom criterion measures are available 
the weighted scores are as close as possible to their actual criterion measures. 
The method commonly used is called the multiple-regression technique. In 
this procedure a set of weights corresponding to criterion categories arc 
applied to the predictor variables. There are as many weights for a given 
criterion variable as there are predictor variables, and as many sets of these 
weights as there are criterion variables. 

It should be remembered that these cutting scores or weights, as the case 
may be, are of no value for the experimental group, since their criterion 
measures are already known. These procedures are used on the administra- 
tive entities as a basis for predicting their success prior to participation in job 
activities. 

Of the two methods just considered, it is probable that the latter is by 
far the best, both theoretically and practically, in spite of the fact that the 
cutting-score method has been extensively used, often by the employment 
of multiple cutoffs successively applied. One of the assumptions in the mul- 
tiple cutoff procedure is that a certain minimum amount of ability is required 
to function in a particular criterion category so that no matter how good the 
person in other functions, these cannot compensate for inadequacies in a par- 
ticular function. The method of linear combination, on the other hand, 
assumes that for a specified criterion activity, superior functioning in one of 
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the predictor attributes may compensate for inferior functioning in another. 
It is true that the former assumptions may apply in some cases. However, it is 
Possible to introduce certain interaction variables into a predictor system 
Which have much the same advantages as multiple cutoffs and are more easily 
handled from a mathematical and computational point of view. For this pro- 
cedure the weight assigned to a given variable is not based solely on that 
variable but is a function of other variables as well (Horst et al., 1941). 

It must be remembered that the variables chosen for use in prediction 
represent hypotheses which constitute the basic framework of the selection 
and classification models. The experimental group of entities, together with 
the predictor and criterion measures available on them and the determination 
of optimal procedure for using the predictor variables, constitute the verifi- 
cation part of the model. For the experimental group, methods are available 
for determining how accurate the estimates of the criterion measures based 
On the predictor measures actually are in terms of the observed criterion 
Measures. These statistical tests of significance indicate the accuracy or 
Validity of the hypotheses set forth. If the hypotheses are regarded as accu- 
rate enough for practical purposes, the procedures may be applied to admin- 
istrative entities. These will be persons who are accepted into the system to 
Teplace those who leave, or to fill jobs created by an expanding system. 
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THIS CHAPTER DEALS WITH THE DESIGN AND | =| 
development of a class of materials that can be used to | | 7 
support various performances required of human beings | 8 | 
in system jobs. They are given the general name of job | | 
aids, and they include such things as printed instruc- 
tions, check lists, diagrams, and the contents of technical 
manuals. Without such aids, the human performer 
would be required to retain in his memory a much larger store of models 
and to learn many more procedures and identifications. Thus, if job aids were 
not used to support such job performances, system technicians would have 
to be selected according to more stringent standards, and traine 
periods of time, because job aids, training, 
methods for obtaining job performance. 
Obviously, then, the design of job aids must take place after human tasks 
and jobs of the system have been decided upon and in conjunction. with 
decisions about how to select men and how to train them. Job aids to be used 
for supporting human performances during System operations may range in 
complexity from simple printed check lists through technical manuals to 
actual pieces of equipment affording visual or auditory displays. In a Strictly 
engineering sense, they are simple to develop and produce. Yet they can and 
do have profound effects in reducing the complexity 


) of selection and training 
procedures necessary to achieve the required performances from human 
System components. 


The psychological problem posed by the design of job aids is here 
conceived as one of stimulus control of human performance. How can stimulus 
situations be arranged so as to produce optimal performances by controlling 
human behavior? It is recognized that this question cannot be sensibly 
answered by attempting to categorize the bewildering variety of human per- 
formances in systems. But it may be possible to classify the varieties of 
change in performance that can be brought about by the introduction of job 
aids. The problem of defining such classes of change, and of relating them to 


characteristics of the performance aid, defines in a general sense the area of 
Scientific questioning discussed in this chapter as a challenge to future psy- 
chological research. 


d over longer 
and selection are complementary 
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AIDS TO 
JOB PERFORMANCE 


J. Jepson Wulff 
and Paul C. Berry 


Within the process of man-machine system development, the term job aid 
has a special meaning. Since that special meaning cannot be given without 
some preliminary discussion, however, we begin by providing a preliminary 
definition in common-sense terms: 4 job aid is something which guides an 
individual's performance on the job so as to enable him to do something 
which he was not previously able to do, without requiring him to undergo 


complete training for the task. : 
An example of a job aid with which many people are familiar is the 


accompanies a new bicycle or an unfinished furniture kit, 
emble the parts. The task of assembling a 
the purchaser has never before encountered. 
paper together with his previously acquired 
using a screwdriver, and the like), a new 
y lengthy period of special training. 
deficiencies exhibited by such 
that there is room for improve- 


Piece of paper which 
telling the purchaser how to ass 
Child's bicycle may be one which 
Yet with the aid of a piece of 
Beneral skills (reading directions, 
Performance can be produced without an 
Almost everyone has had experience with the 
Printed instructions and will recognize at once 


ment in the construction of such job aids. ore, . 
Two kinds of psychologists are commonly concerned with job aids. The 


first is the psychologist engaged in man-machine system development. He is 
interested in job aids because they provide him one means for obtaining the 
uman performances that are necessary for system performance. The second 
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is the research psychologist. An effective job aid is an article which serves to 
elicit and control human behavior, and this of course is central to the research 
psychologists domain of interest. 


Objectives of the Chapter. This chapter is directed primarily toward 
the research interest of the psychologist. Its aim is to show the relation 
between the practical job-aid problem facing the system developer and some 
of the questions about human behavior faced by the research psychologist. If 
this chapter achieves its objective, after finishing it the reader should be able 
to identify some major research problems that require solution to enable the 
psychologist in industry to construct better job aids, and to relate these 
research problems to the subject matter and to the common concern of 
research psychologists. 


Contents of the Chapter. The body of this chapter is divided into two 
main parts. The first is designed to provide useful background information 
about the development and use of job aids in man-machine systems. In reading 
this part, the reader should recall that our aim is not to teach him how to go 
about building a job aid of his own, but rather to foster a research capability. 
The information that is presented about the development and use of job aids 
is provided for the purpose of defining the practical requirements which basic 
research must sooner or later satisfy so that the psychologist in industry can 
improve his capability to develop effective job-aid materials. 

The second part of this chapter is concerned with the psychological 
aspect of research and development on job aids. This part is focused directly 
upon the main objective of the chapter. It is designed to relate the job-aids 
problem to psychological principles and findings with which the research 
psychologist is familiar, and thus to relate the practical problem of producing 
job aids to the research psychologist’s major interests. It is intended to provide 
the reader with an orientation that will permit him to carry out his own 
research and development work. It is not a review of what has been done in 
the field: in fact, it needs to be borne in mind that very little research has 
been done that has been focused specifically on the problem of developing 
job aids. 


f JOB AIDS IN SYSTEMS 


The users of complex systems naturally think of job aids as essential 
adjuncts to system hardware, necessary for its operation and maintenance. 
To think of job aids merely as adjuncts, however, can be misleading. It is more 
useful to conceive of job aids as integral parts of the total system, produced 
during system development to provide for the reliable operation of the total 
man-machine system in the field. 
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The Role of Job Aids in System Operation 


A job aid is an item which is delivered as part of an operational man- 
machine system along with the hardware and personnel. A job aid may be 
printed on paper, it may be a simple device, or it may be a motion picture. 
But whatever form it takes, its purpose is to foster some performance that is 
required in the operation or maintenance of a system. Once a job aid is 
delivered for use by system personnel, it is normally not expected to be 
modified in the field. Therefore, although the research psychologist will be 
interested in the job aid in system operation. his major interest is likely to 
center upon the research and development that can be brought to bear upon 
its effectiveness before the item is manufactured and the manner of its use 
can be determined. 

The principal thing that the research psychologist needs to know about 
job aids in the field is the manner in which they are properly evaluated. A job 
aid is delivered as part of an operating system for only one reason—to support 
Some performance on the part of an operator or technician that is needed to 
keep the system "on the air." This means that job aids are given their final 
evaluation by determining whether or not they actually support the job per- 
formance that is required. Further, any such performance evaluation must be 
accomplished under the actual operating conditions that will obtain when the 
job aid is used with serious purpose. Under proper conditions, job aids will 
not be evaluated in terms of their appearance nor by expert opinion, but rather 
by measuring observable human performance under representative field condi- 
tions. To know how job aids are evaluated is to know the objective of job aid 
development, and thus to know what practical purpose research on job aids 
must serve. Research must be focused upon the development of techniques for 
Obtaining job performance by means of job aids and upon the development of 
techniques for producing the performance under the specific kinds of condi- 


tions that will obtain in the operating system. 


Job-Aid Development in System Development 


It is primarily during the development stage that research findings can be 
brought to bear upon the job-aid problem. Therefore, in order to define the 
development task which requires supporting psychological research, let us 
turn to a description of the way in which job-aid development is integrated 
into a man-machine system development cycle. . 

" Man-machine systems are, of course, developed in many different ways 
if one considers the detail of individual development cycles. However, there is 
a pattern to what is done in system development which can be abstracted in 
8eneral form, and which can be said to be typical of development cycles. A 
general description of such a development cvcle is presented here to provide 
the context for talking about job-aid development, and it is presented sche- 
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matically in Figure 8.1. The figure shows that the development cycle is parti- 
tioned into two major parallel functions, a hardware development cycle and a 
personnel development cycle. The personnel development channel is separated 
from the hardware, not because it provides logically distinct contributions to 
the system, but rather because the problems associated with the actual provi- 
sion of one or the other are so different. Thus the hardware channel accounts 
for the development of all the hardware means needed to implement system 
performance, while the personnel channel includes all the activities needed to 
produce those functions allocated to personnel. 


HARDWARE DEVELOPMENT 


hardware 


SYSTEM INTEGRATION AND TESTING 


PRELIMINARY DESIGN 
(In response to system 
requirement as an input) 


SELECTION 


All 
f TRAINING materials 
Performance MATERIALS 
= | (tobe, | DEVELOPMENT needed for| «5 
E traine! training = Trained 
Requirement u Requirements for = personel 
for personnel I training in E 
performance job aid use 
obtained JOB AID Job aids Job aids 


DEVELOPMENT 


by position 


1 
i Hardware information 
from Hardware Development 


Figure 8.1 Schematic description of a system development cycle 
emphasizing job aid development. 


The job-aid development task is embedded within the personnel develop- 
ment cycle. In this figure, the job-aid development task is defined in terms of 
the inputs it receives during the development process and the outputs which 
are required of it. As shown in the figure, job-aid development is preceded by 
job definition (design), which results in a description of all the objective 
performance requirements for all the operator and technician positions within 
the operational system. Job definition further breaks down cach job into a list 
of those performances which will be obtained by selection, those which will 
be obtained by means of training. and those which will be obtained by means 
of job aids. 
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The last of these is the major input to the job-aid development unit. It is 
therefore worthwhile to examine the nature of this input in detail. Specifically, 
this input will be made up of a series of "orders" for job aids, each order 
implying a need for a job aid. An order will be identified with respect to the 
position that it is associated with, and it will define in objective terms the on- 
the-job performance that must be fostered by the job aid. An order is thus, in 
in a sense, a description, not of the details of the needed job aid, but of the 
way in which the job aid will be evaluated in the operational situation. To be 
complete, the order must also specify the relevant capabilities which can be 
assumed on the part of the man who will use the job aid, and it must specify 
the conditions under which the job aid will be employed that will influence 
the physical characteristics of the job aid. 


The Nature of Job-Aid Development. Since the job-aid development 
unit operates upon each order in much the same manner, we shall undertake 
to describe the operation of the unit for a single typical order. Ignoring for 
the moment the other inputs which must be made to the job-aid unit, let us 
turn to a description of the major output of the unit, because this defines the 
Principal objective of unit operation. That output is a job aid which will 
actually foster the job performance specified when it is employed by a tech- 
nician who has the relevant capabilities (also described in the order). This 
Output is, as shown in the diagram, a major final output of a system develop- 
ment cycle, just as hardware and trained personnel are final outputs. The 
Output of the job-aid unit is something which appears in the operational 
Situation as an integral part of the operating system, necessary to provide 
accomplishment of the system mission with a given reliability. 

Some of the secondary inputs and outputs of the job-aid unit also need 
to be considered here. In order to respond to an order for a job aid to be 
developed, it is necessary to collect detailed information about the hardware 
Upon which the technician is required to perform; and this calls for an input 
Of information from the hardware development cycle. Further, in the actual 
development process, interaction is required with the selection unit, and the 
System training and test functions, although these are not shown in the figure 
(compare Chapter 13). An important secondary output which must. be 
Obtained is one which goes to the training-materials development function, 
carrying the requirements for training in the use of the job aid. This output 
'5 an important one to note because, by providing it, we take particular notice 
Sf one characteristic of job aids: it is not necessary for a job aid to be con- 
Structed so that it can be used with no prior training at all. It is only required 
x at a job aid reduce the amount of training below that which would be 
“quired if no job aid were provided. V 

In the Mim indie system development, the job-aid develop- 
Ment problem is simply this: the job performance to be supported by job aids 
.USt be identified, and then a means must be found to proceed from the 
Initia Statement of the performance that must be produced (together with the 


278 - Aids to Job Performance 


description of the hardware and the prior capabilities of personnel), to the 
production of the actual materials that will guide the performance appro- 


priately. 


What Does the Job-Aids Development Specialist Do? 


By describing briefly the work of the job-aids specialist within the devel- 
opment cycle that has just been depicted, it will be possible to call attention 
to the kinds of development actions that require supporting rescarch. The 
developer of job aids contributes to two of the functions shown in Figure 8.1: 
The design unit and the job-aids unit. The major development actions that 
he will take are described below. This account should not be taken as a 
prescription for job-aid development, however. The order of the actions, and 
even the need for some of them, will differ from one system development 
problem to another. The list is simply intended to be a representative one. 

1. The job-aids specialist is first involved in system development within 
the unit called “design.” As shown in the figure, the function of this unit is to 
prepare job definitions in terms of the performances required and to break 
down each job into requirements for selection, training, and job aids. In order 
to accomplish this three-way breakdown, it is necessary to draw upon the 
talents of selection specialists, training specialists, and job-aid specialists. This 
kind of cooperative endeavor is necessary because the determination of how 
to obtain performance must be based upon consideration of the state of 
existing technology. Thus, it is inefficient to mark a particular performance for 
training if it is within the state of existing technology to accomplish it by 
means of a job aid, and it is also inappropriate to mark a performance for 
development by means of job aids if there is no known job-aid technique that 
can be employed to accomplish it. Therefore, in deciding upon the means that 
will be employed to obtain all of the performance for a given position, it will 
be necessary to foresee where current technology will allow the use of job aids, 
where it will allow the use of a training method, and where selection can be 
employed. One end result of this effort should be *orders" for the development 
of job aids which will be passed on to the job-aids unit for processing. 

2. Once an order has been given to the job-aids unit, the first major step 
to be taken within this unit will be to prepare a very specific description of the 
performance that must be fostered by means of the job aid. There are two 
different cases to consider in undertaking this step. The first is the case in 
which the detail of the actions which must be supported by the job aid is 
completely determined by the equipment in the system or by other factors 
external to job-aid development. In this case, the action to be taken within this 
step is simply that of finding out what the detailed performance is and of 
describing it. The second case to be considered is that in which there are 
alternative ways that could be employed to achieve the performance that is 
specified in the order. Typically, alternative performances are possible when 
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the human activity to be fostered involves problem solving or dynamic decision 
making. Therefore, this step in job-aid development may be a rather difficult 
One; it may be necessary to describe and then to choose among a number of 
different ways of leading the technician to carry out the required performance. 
In either case, at the end of this step in job-aid development, there must be a 
detailed description of the actions to be taken by the technician, and these 
actions will be the ones to be supported by the job aid to be developed. 

3. A job-aid medium must be selected. Given a detailed description of 
the actions which must be supported by the job aid, and given a statement of 
the on-the-job conditions which must obtain, the best medium for presenting 
job-aid information must be chosen. The medium chosen may be a printed 
Page, an auditory guide, a motion-picture film clip, an automatic information- 
Selecting device, or some other medium. Whatever one is chosen, it must be 
Compatible with the operational requirements of the job aid and of the job. 

4. The detailed content and format of information to be presented by 
the job aid must be developed and prepared in a form suitable for use in the 
medium selected for presentation. The specific stimulus materials that will be 
the job aid must be designed, and whatever they may be, they must be cal- 
Culated to support the specific detailed job performance required. 

5. The prototype job-aid materials must be subjected to evaluation and 
Must then be refined and specified for production. 

6. The job aids must be produced and delivered as part of the opera- 
tional system. They must also be supplicd to the training program, where 
technicians under training will learn to use them. 

7. The training requirements generated by the development of the job 
aids must be specified and forwarded to the training materials development 
unit, where materials will be prepared for training technicians to use the 
job aids, 


© THE NATURE OF JOB AIDS 


Now that we have described the system development context within 
Which job aids are designed, we can return to the question of definition. As 
We have pointed out. a job aid may be defined in terms of its use in an 
Operating system, but such a definition is not of greatest utility to the research 
Psychologist, Instead, the characteristics of a job aid are here conceived in 
terms of the problem of job-aid development. 

A job aid is an item whose purpose is the support of performances by 
SYstem Personnel, which are necessary for over-all system perfor mance. It is 
developed as an alternate to training as a means of obtaining the necessary 


D, PREA " Ws 
D erformances, and is therefore designed deliberately to complement training. 


he job aid is developed in response 10 a requirement which specifies the 
Performances that must be supported and the conditions under which these 
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performances must be obtained. It is an end product of a system development 
cycle which is delivered as part of an operational system. 

This kind of definition can be useful to the research psychologist because 
it identifies the situation in which research findings must be brought to bear. 
That is, the definition states that the research findings must be useful for a 
situation in which development starts with a performance requirement, is 
integrated with training development, and ends with the delivery of an item 
that will be assessed in terms of the performance which it is supposed to 
support. The definition also implies that job aids are articles which are 
developed as job aids; it is not useful to classify all items which *look like" 
job aids as job aids, if they have been developed for other purposes. 

Referring back to the stages in job-aid development outlined in the 
previous section, we can identify two particular points at which support from 
basic research is to be sought. The first (step 2) is in connection with the 
detailed description of performances that must be supported by the job aid 
(compare Chapter 6). When the job-aid developer must prepare such a per- 
formance description for a problem-solving or decision-making activity, he is 
likely to find that the research literature on these topics has many relevant 
findings. The second major step which creates a need for research is that 
concerned with the development of stimulus conditions to be presented by 
means of a job aid in order to bring about the required performance (step 4). 
Research on the relationships between stimulus input and performance output 
is of central interest to the human learning theorist. The psychological research 


Tequirements generated by these two job-aid development steps are described 
in a later section of this chapter. 


Some Examples of Job Aids 


Before going on to a discussion of the psychological problems associated 
with job-aid development, it will be useful to examine some examples of actual 
job aids. There are two major classes of job aids which we need to consider 
particularly. The first class is composed of job aids which foster the same 
performance steps that would be taken by the technician if he were to be 
trained to do the task without the use of job aids. A common example of a 
job aid of this kind is a lubricating chart. Such a chart calls for essentially the 
same activities on the part of the technician as would be exhibited if he were 
to learn the task “by heart.” The advantage of the job aid in this case is that 
it obviates the requirement for training without Sacrificing performance capa- 
bility, and perhaps with a gain in performance reliability. In other words, à 
somewhat lengthy sequential routine, rather than having to be memorized as a 
"model," has simply been provided to the technician so that he can refer to 
it on the job. 

A second class of job aid i« composed of aids which make it possible to 
obtain the desired performance without requiring the user to go through all 
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the steps that he would normally take if he were to perform the task after 
having learned it. An example of a job aid of this kind is a transformation 
table. Job aids in this class are usually prepared by experts who carry out all 
of the detailed performance steps required and then encode the results of their 
labor in such a way that the user need only perform the first and last steps 
in the sequence. 

Thus, in using a transformation table, one must perform only the steps of 
identifying the initial given information and of identifying the answers by 
decoding the table; it is not necessary to go through all the intermediate steps 
of calculating, which one would normally perform if the job aid were not 
available, 

The diagram shown in Figure 8.2 is part of a troubleshooting job aid for 
4 complex electronic equipment. It is an example of the first kind of job aid, 
the kind which supports the same job performance as the performance which 
Would be carried out if training were used instead of the job-aid method. In 
the case of this example, the circles in the diagram identify all of the indicators 
Which the technician must read out as his first step in the troubleshooting 
Procedure. The circles thus support the read-out performance in the place of a 
memorized list. The boxes in the diagram identify the functions in the system 
to which a malfunction can be isolated by means of the given read-outs. 
Therefore, like circles, boxes make it unnecessary for the technician to com- 
mit information to memory—in this case, the list of functions. The fact that 
the boxes and circles are arranged in a certain order, which is shown by the 
arrows Connecting them, is also useful to the technician. The arrows connect- 
Ing the boxes and circles show the information flow in an Operating system, 
and this is information which the technician must use to determine where to 
find a malfunction. These relationships are much more difficult to page 
than the list of functions or the list of read-outs, and it is primarily to een it 
Unnecessary for the learner to remember these relationships that E aid is 
Provided. In general, the job aid materially reduces the iiy iu 

at is required to teach a technician to troubleshoot the system. (it ps be 
realized that only a portion of the total troubleshooting ip nn is nl in 
© figure. The total diagram is a great deal more complex, but rng af ^d 
a Same Principles as the portion shown.) In using this job aid, the tech- 
cian goes through the same procedures as those he Sone oec he had 
Memorize all of the information contained in the job aid in order to carry 

'S troubles ing. à ‘ : i 
„Now os an example of the second kind of ye n xs which 
Changes the job performance of the technician, as compared wit the case in 
Which Jo sue 3 duty without a job aid, on the basis of train- 
ing, bu k eos perform the s N ara is a simple example of this kind of job 
a P P. Fbst fable used in troubleshooting. To use the 

'àgram sh qe 8.3, the technician ust have gathered some infor- 
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Figure 8.3 Example of a job aid which reduces the performance 
Steps, as compared with the steps which must be taken by a technician 
trained to perform without a job aid. (From Psychological Research Asso- 


Ciates, 1961.) 


guessed which function, or box, in the system contains the malfunction. He 
then enters the table from the top and reads the column below the suspected 
malfunctioning function-box. In this column he will find a pattern of three 
Tead-outs which he must obtain in order to confirm his guess about the loca- 
tion of the malfunction. The technician who uses this table to check out his 
Toubleshooting guess must perform significantly less work than the technician 
Who Carries out the same task without the table. That is, if the table were not 
Available, after the technician had made a guess about the bad box, he would 
a plan by which he could check his guess. 
Orking out such a plan involves a considerable amount of time and effort. 


© Preworked plans which are shown in the table were fien den 
engineer y s working with the equipment for that purpose. 
His Whio: spentseyerat days ble and need not be repeated by the 


` Work has been encoded in the ta d 
“chnician each time he wishes to carry out one of the troubleshooting checks 


“Scribeq in the table. 7 —— 
twe he two examples of job aids which have been des plify the 


di > classes of job aids we have defined. However, job aids appear in many 
ferent forms and are used for many different purposes. The reader should 
and a 


en have to work out for himself 
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not assume that these examples constitute a representative sample of all of 
the varieties of job aids in use. Related discussions of job aids and their devel- 
opment can be found in treatments by Hoehn and Lumsdaine (1958) and by 
Miller (1956). 


Varieties of Tasks Supported by Job Aids 


We have said that job aids are used to support the performance of many 
different kinds of tasks. It is therefore natural to want to categorize job aids 
in some way that will identify the tasks for which they are useful. Any attempt 
to classify the tasks for which job aids are used should, however, serve some 
useful purpose; such an endeavor should do more than satisfy a natural human 
requirement for order. One reason for classifying tasks might be to assist in 
identifying the situations for which job aids should be developed for new 
systems. We might attempt this by noting that job aids are used with good 
effect to support the performance of lengthy procedures—especially when the 
lengthy procedures are used only infrequently and would otherwise be subject 
to forgetting. Job aids are also used to foster stimulus-identification tasks, as 
when pictorial presentations of oscilloscope read-outs are used in maintenance 
handbooks to provide the technician with a standard against which he can 
compare his obtained read-outs to determine whether a signal is good or bad. 
Job aids are also used to support problem-solving tasks. These, as described 
above, are sometimes used to reduce a problem-solving task to a fixed 
procedure by the use of a transformation table. Job aids are also used quite 
frequently to replace memorial inputs, as we have also exemplified above. 
If we continue attempting to identify the situations for which job aids are 
useful, however, we will sooner or later come to tasks for which job aids arc 
sometimes useful, but for which it is at other times impossible to construct 
useful job aids. We will also come to tasks for which job aids are not used. 
An outstanding example of a class of tasks for which job aids are not used 
includes those tasks which involve complex motor performance. 

A little reflection about those tasks for which job aids are not used will 
shortly suggest the conclusion that they are not used simply because no one 
has yet developed job-aid techniques which are suitable. In fact, just a few 
years ago, some of the tasks for which job aids are now used would necessarily 
have fallen into the category of tasks for which job aids are not used, simply 
because the development of appropriate job-aid techniques had not yet taken 
place. The truth of the matter is, job aids are used whenever a way can be 
thought of to develop a job aid that will effectively support some performance 
demanded of a technician; job aids are not used when a way of constructing 
them cannot be conceived. Perhaps the time will come when job aids will be 
developed to support the performance of complex motor tasks. Right now; 
however, it is not known how such material could be presented. But the 
realization that one might at some time be able to do so points up the fact 
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that the categorizing of tasks for which job aids can be used is a dangerous 
practice. Such a list can never be comprehensive and must continuously be 
revised as the technology of job-aid development improves. Later on, we will 
discuss a way to categorize tasks for a quite different purpose—namely, for 
the purpose of deciding how to build a particular job aid rather than for the 
purpose of deciding whether or not a job aid might be useful. For the latter 
Purpose, it is much more reasonable to use criteria such as the following when 
deciding whether or not to use a job aid: 

1. The use of a job aid to support the performance of a task should be 
considered if a job aid can be projected that will actually support the perform- 
ance. If no job aid that will provide the required performance support can be 
described, there is no need to consider the matter further. 

2. If a job aid which might be used can be described, the next considera- 
tion should be whether or not its use will actually reduce the amount of 
training time that is required. Sometimes it will take so long to train a tech- 
nician to use a particular job aid that it would be more economical, in terms 
of training time, to teach him to perform the task without the use of the job 
aid. There may be no gain in using a job aid if the training time required is 
Not lessened by using it. 1 

3. If the use of a job aid will save training time, one should next con- 
Sider the cost of developing and using the job aid. If the cost in terms of 
dollars is much greater when the job aid is used than the cost of the training 
approach, consideration should be given to abandoning the proposed use of 
the job aid. . ' 

4. Finally, if there is an available job-aid technique that will support the 
required job performance at a saving in training time, and if it costs less than 
training, one must ask what effect the use of the job aid will have on the 
reliability of performance. Most often, job aids will improve the reliability of 

uman performance over the reliability that is obtained when training is 
employed, However, deliberate attention should be paid to this fact, because 
à job aid should not be used if it will degrade the percentage of times that the 
Uman performs correctly below that which can be tolerated within the system. 

When criteria like the above are used to determine whether or not to use 
a job aid. there is no need to categorize the uses of job aids. Rather than 
Categories of uses for job aids, the job-aid expert needs a large repertory of 
JOob-aiq techniques, so that he will be able to think of a job aid to match almost 
any task required. In the last section of this chapter we will discuss what is 
*equired to build a large repertory of job-aid techniques. 
| We have considered how requirements for job aids ate. developed, how 
Job aids themselves are developed in response to these requirements, what job 
aids look like, and how to decide when to use a job aid. With this background 
SStablisheg it is appropriate now to turn to consideration of those aspects of 

© job-aid problem which relate directly to the content and approach of 
Psychology. 
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Two major aspects of job-aid development are of interest to psycholo- 
gists. The accomplishment of both of these requires the development expert 
to utilize presently available psychological information, and both steps also 
call for further research by psychologists to develop new information which is 
needed to improve our present capability to develop job aids. The first is the 
problem of describing in detail a selected set of performance steps which must 
be supported by a job aid when there are alternative ways of accomplishing a 
task. The second is the determination of the stimulus characteristic of a job 
aid, such that the job aid will provide appropriate stimulus control to insure 
that the technician will carry out the performances which must be supported. 
We will now consider the research problems associated with these two devel- 
opment problems. 


[S RESEARCH PROBLEMS 


The first problem to be discussed—that of selecting and describing à 
detailed sequence of performance steps to be fostered when there are alter- 
native ways to accomplish a task—is one common to both the development 
of training materials and the development of job-aid materials. However, in 
the development of job-aid materials this problem is of even greater signifi- 
cance than when training has been selected as the method for obtaining job 
performance. This is true because job aids must necessarily be constructed to 
promote the performance of very specific actions. Training, on the other hand, 
can be carried out in such a way that considerable latitude is left on the part 
of the operator or technician when it comes to doing the job. Thus, when à 
training approach is applied, it may be possible to give considerable basic OF 
background training with respect to a decision-making performance Of 
problem-solving performance, such that the technician can employ ingenuity in 
selecting his own methods of solution when he meets with a real problem on 
the job. In general, this kind of latitude is not possible when the job-aid 
method is used. The specific performance to be fostered must, therefore, be 


decided upon in advance and must be described in great detail before the job 
aid is constructed. 


Identifying Performances for Job-Aid Support 


We have already encountered one example of job performance which 
poses a problem when it comes to defining the specific actions which must be 
fostered by means of a job aid. Thus, in order to construct the job aid shown 
in Figure 8.2, it was necessary to describe the desired actions in detail. The 
job aid in Figure 8.2 is designed to support troubleshooting performance for a 
complex electronic system. Trou5leshooting is a problem-solving performance; 
and for many systems there are alternative ways of troubleshooting. For 
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example, one kind of performance routine might call for the technician to 
Start by taking a reading of system performance at an indicator on the output 
side of the system and then to work backward through the system, step by 
Step, taking additional readings which would gradually partition the system. 
Eventually, by this process, he would narrow down the location of the mal- 
function to a replaceable component. An alternative routine might be to 
gather information from many read-outs of system performance as the very 
first step, for the purpose of isolating the malfunction to a rather large func- 
tional area by using the pattern of read-outs as a syndrome. If this approach 
is taken, the indicators to be used for this first step must be carefully selected, 
So that isolation to a functional area will be possible. As a second step, the 
Process might be repeated within the functional area identified in the first step, 
and so on until the malfunctioning replaceable component is identified. Other 
Performance routines could be described, but these two will suffice to demon- 
Strate that alternatives are possible and to make the point that a particular one 
Must be selected before a job aid can be constructed. 

Figure 8.2 is based on the second of the two performance routines de- 
Scribed above, and it therefore contains only that information necessary for 
Carrying out that kind of troubleshooting. Had a different troubleshooting 
TOutine been selected, the job aid in Figure 8.2 would be structured quite 
differently. The conclusion here is that when decision-making or problem- 
Solving behavior is called for, there may be alternative ways to carry out the 
Process, and a particular one must be selected and described in detail before 
an appropriate job aid can be constructed. This creates a research problem. 
Specifically, we need to know something about problem-solving and decision- 
Making behavior as it is carried out by human beings and as it might be 
Carried out by human beings. Thus, not only is it necessary to be able to 
describe in detail some kind of problem-solving or decision-making behavior 
Which will work. but it is also necessary to be sure that the routine which is 
Prescribed is one which human beings are capable of carrying out. Only then 
can a job aid for a decision-making performance be constructed. 

Thus far, the terms “decision making” and “problem solving” have been 
Used rather loosely. The phrases commonly refer to processes internalized in 

© human brain by which information is processed and to which the more 
8eneral term “thinking” is also applied. In this discussion, the reader should 
NOt seize upon what may seem to be inappropriate usage of these terms, but 
should remember, rather, that these terms are used here to refer to a class of 
internalized processes which meet the following criteria: they must be spe- 
Cifiable on the input side by an objective description of the stimulus situation 
hich initiates the process; they must be described on the output side by an 
Jective description of a correct answer Or state of affairs which must be 
wc hieved when the process is complete; and the process which mediates be- 
ae input and output must be one which: is not fixed, but which can be 
Med out in two or more ways with at least occasional success. 


Lo —————————————————— 
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For thinking tasks whose structure is fairly simple and quite explicit, 
logical analysis of some of the available procedures is possible, and it isa 
comparatively straightforward task to select an efficient rational procedure. 
For example, alternative procedures may be evaluated in terms of the number 
of steps required to carry them out or the expected cost or time implied by 
the steps. For many complex problems, however, there does not yet exist any 
rational procedure by which the information can be processed. In some cases 
a perfectly rational procedure may be conceivable, but it is also obviously 
impractical. It is therefore necessary to fall back on temporizing procedures 
that will work some of the time or may be preferable to no action, but which 
lack the elegant assurance of success that characterizes rational approaches to 
simpler problems. 

Let us take, for example, the problem of specifying moves in a game of 
chess. In principle, one could examine every possible move, every possible 
reply of one's opponent, every possible move after that, and so on. But even if 
computerized assistance were used, this task would take far longer than à 
chess player's lifetime. This means that a strategy must be adopted which will 
inevitably leave many possible moves unconsidered. The devising of such 
strategies and the evaluation of alternative ones becomes an extremely difficult 
task. And this is a difficulty which exists quite apart from our consideration of 
the limitations of the human being who must provide this performance. The 
difficulties we have mentioned are logical difficulties in the nature of the 
problem and in the information processing required to solve it. 

Now let us suppose that a rational strategy of some sort can be concocted 
for approaching a particular information-processing problem; it still remains 
to specify this strategy in terms of the repertory of information-processing 
abilities of the human operator. This is undoubtedly our area of greatest 
ignorance. While the processes of human thought were once considered the 
central subject matter of psychology, remarkably little progress has been made 
in analyzing them. Certainly we are utterly ignorant of the physical and chem- 
ical bases of thinking, and we can say little more about the thought process 
than that it is probably located in the brain. It seems more optimistic tO 
attempt to describe human capabilities in terms of the information processes 
of which human beings are capable. But this too is an area about which we 
have so far only the most rudimentary knowledge. Early attempts to study 
thinking by introspection rapidly revealed that men are largely unaware of the 
processes by which they think. Research since then has had to be content with 
examining only some of those peripheral conditions which may facilitate Of 
inhibit the final output of a thinking process. 

It can be seen, then, that one research problem which is particularly 
important for the development of job aids for decision making and problem 
solving is the development of ways to determine good strategies for carrying 
out these activities. A second.research problem, and one which is more 
relevant to the interest of the psychologist, is the determination of what func- 
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tions (either primitive or learned ones) human operators may have in their 
repertories. This determination is essential to the job aiding of the specific 
steps the human is to utilize in carrying out his strategy. At the same time, this 
determination will have considerable bearing on the selection of strategies that 
are feasible of accomplishment by human performance. 


The Programming of Thinking Activities 


Let us now turn to consideration of ways in which these research prob- 
lems might be attacked. This discussion will center on the use of computer 
techniques which have only recently been developed and which promise to 
provide new methods both for the rational solution of information-processing 
problems and for examining the processes involved in thought. In considering 
this research approach, it should be kept in mind that the program for the 
Computer serves a function comparable to that of a job aid for human being. 
In each case, once the desired performance has been specified, a strategy must 
be developed that is capable of solving the problem and which can be built up 
from the more primitive actions of which the man or machine is capable. As 
the programmed instructions call out machine operations, so must the elements 
Of the job aid call out human operations. Thus, in a sense, a job aid is to a 
human as a program is to a computer. 


Use of the Computer in the Study of Thinking. The invention of the 
Computer has given a new boost to the study of thinking. It is interesting that 
it should have done so, because no new discoveries about the nature of thought 
have followed from the use of computers, and computers are not capable of 
activities that could not have been undertaken long before the first computer 
Was built, Rather, the existence of the computer and of advanced computer 
Programs has made undeniable the fact that activities that we would ordinarily 
describe as "thinking" can be undertaken by machines. This has served to 
remove some of the mystical flavor which surrounds this topic. 

It was not long after the use of high-speed computers for problems of 
arithmetic that programmers turned to the exploration of some of the other 
logical possitis oF Tue rene dem particular the use of heuristic 
Strategies for approaching problems of extreme complexity. The results of 


Some of the early pioneering efforts in this field are provocative for several 
reasons, First they indicate that by employing heuristic methods as opposed 
to algorithms, computers may be set to work on problems in areas heretofore 
Considered solely the province of man’s imagination. By processes which 
remain completely explicit (for computers cal deal with no others), com- 
Puters have been set to work making economic choices, playing chess, and 
Solving theorems. (See also the discussion of the capabilities of computers 


Chapter 3.) x | ] " 
The second provocative result of heuristic programming of computers is 
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that computers thus controlled sometimes make mistakes. This, of course, is 
the inevitable consequence of the rejection of the algorithms. Now it is inter- 
esting to note the kinds of mistakes that are made and the relation of these 
limitations to the type of information processing employed. Computer pro- 
grammers have begun to compare the strengths and weaknesses of their 
artificial thinkers with those exhibited by human beings. This effort was under- 
taken at first simply to see if any machine method might be devised which 
would reproduce some of the characteristics of human thought. Gradually the 
programmer psychologists have become emboldened to make more direct 
comparisons between the processes that they know exist in the computer with 
those that it appears might exist within the human head. Thus the develop- 
ment of heuristic information-processing programs for computers offers not 
only an approach to the solution of complex problems in a rational manner 
(or at least an explicit manner), but also a language and a way of thinking 
that may then be applied to the study of human thought processes. 

Obviously, the mechanisms within the head do not even faintly resemble 
the hardware employed by computers. Just as obviously, some sort of infor- 
mation processes are used within the brain. Because of the extreme flexibility 
of digital computers, it may be possible to make the computer imitate the 
information flow (although not the physical mechanism) that occurs within 
the brain. Certainly, human thought exhibits many characteristics of radically 
heuristic programming: it is extremely rapid for many tasks, is capable of 
handling problems of great complexity, and yet is not always highly reliable. 
With the development of computers more specifically designed to imitate some 
of the characteristics of human thought, it may prove possible to pursue these 
comparisons much further. Already there is available a whole new approach 
that may be applied to the study of thinking, and that is to ask (with or with- 
out computer assistance) the question: what sort of information-processing 
heuristic would result in the operating characteristics actually observed in 
human thinking? 

The techniques of the study of information processing remain so rudi- 
mentary that we can at present only hope that it will prove possible genuinely 
to obtain some new understanding of human thinking by this route. At present, 
the development of computer techniques in thinking is following two lines of 
development: simulation and artificial intelligence. Simulation refers to the 
construction of programs whose operating characteristics duplicate those of 
human subjects, with a view to making some inference about the nature of 
the information process used by the human being. Artificial-intelligence 
projects are concerned simply with using computers to think as best they can 
and, if possible, better than humans do. In this latter venture the interest is not 
in duplicating human errors but in providing thinking functions in the most 
efficient way possible. 

Possibly much of the useful contribution of computer technique to the 
study of thinking will be the contribution of a language by which psychologists 
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can describe their impressions of human thinking. These need not involve any 
electronic equipment at all. Those who debug (that is, troubleshoot) com- 
puter programs become fairly skillful at guessing in what processes faults may 
lie from inferences about the errors produced in machine output. This is just 
the position the psychologist is in: human output is readily visible, and process 
is not. By asking what sort of program might produce such outputs, psycholo- 
gists may be able to arrive at useful hypotheses about human thinking. 

Before significant gains can be made in the development of job aids that 
will effectively foster problem-solving and decision-making performance, psy- 
chologists must begin to identify through research those human functions 
Which are involved in problem solving and decision making. Probably the way 
to identify these functions is not through the use of computers as research 
tools, but rather through the use of computers as an example which will help 
to keep the research psychologist focused upon the proper question. If one 
Wishes to know what functional capabilities the human is endowed with, the 
Proper object of study is the human. The suggestion made here is merely that 
the human be studied as though his output were the composite result of 
functioning components. The goal of research must be to identify those 
functions which the human can or should employ in problem solving and 
decision making. 

Let us summarize by considering a hypothetical operator performance 
Tequirement for which we might wish to develop a job aid. Our hypothetical 
€xample is a decision-making performance which must be carried out by a 
Submarine officer. His performance is initiated by a signal that one or more 
targets have been detected. The output of his performance consists of four 
Orders: (1) identification of the target or targets that will be attacked; (2) 
Identification of the weapon that will be used against each target; (3) a state- 
Ment of the time at which each target will be attacked; (4) a plan for own 
Ship retirement. In order to “job aid” the performance which must mediate 

etween the initiating signal and the final outputs, We must be able to identify 
8 set of specific performance steps which can be carried out by the officer and 
Which will always bring problem solution to a successful conclusion. Only after 
We have done this can we construct a job aid which will successfully give the 
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beings need to employ in order to carry out a specific task, a list of instructions 
must still be constructed which will, in fact, elicit the required performance 
components in the right order and at the right time. Let us turn next, then, to 
consideration of the research that is necessary to develop basic information 
about ways in which we can obtain some desired behavior by means of 
stimulus control. 


The Stimulus Characteristics of Job Aids 


The role of a job aid as a stimulus complex can be pictured as shown in 
Figure 8.4. In this figure, S, stands for the signal which indicates that some 
specific performance is required and A, stands for the state of affairs which 
must be achieved when the performance (response) has been completed 


TIME 1 


S. Si Ri 
No Ry 
TIME 2 


Stimuli 
presented by 
job aid 


Figure 8.4 Schematic description of performance before and after 
the introduction of a job aid. 


successfully. The figure shows that at Time 1, prior to the introduction of the 
job aid, the technician will not respond with R, when S, 
figure also shows that at Time 1 the technician will be capable of certain 
elementary performances (S,-R;, S.-R., and so on). Further, the figure states 
that at Time 2, when the job aid is introduced, the S.-R, capability will be 
exhibited. Specifically, at Time 2, S, will lead to the selection of the appro- 


priate job aids; and when the job aid has been selected, it will present stimulus 
conditions which, in conjunction, with S,, will elicit R 


of performance capability that existed at Time 1. 


+ is presented. The 


u by way of the elements 
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The symbolic representation in the figure calls attention to the principal 
Tesearch problem to be discussed here. Research is required to answer the 
question, “What kind of stimulus conditions must be afforded by the job aid 
to obtain the required job performance?” Here, then, concern will be with the 
form and content of a job aid; the objective is to develop pictures, words, 
devices, or sounds that will induce a technician to perform some specific 
behavior. When we reach this stage of consideration in job-aid development, 
it does not matter what technique has been used to generate a description of 
the behavior that the job aid must foster. Whatever the behavior is, the objec- 
tive becomes one of developing a set of stimulus materials of some kind that 
will obtain the required performance both reliably and quickly. 


The Need for Various Types of Job Aids. The problem of developing a 
set of stimulus conditions which will constitute an effective job aid would be a 
Telatively simple one if it were possible to formulate a single set of rules for 
doing so. However, it seems rather clear that rules for developing a job aid 
Which might work well in one case are not at all appropriate for another. For 
example, although job aids have been constructed for guiding fixed procedural 
Performance, the same type of job aid will not be useful for other kinds of 
Performance. 

As soon as it is recognized that there must be several sets of principles 
for developing job aids as stimulus materials, then it can be seen that the 
Objective of research must be to create several kinds of information. The job- 
aid specialist must eventually be able to take any job performance and decide 
What class of performance it represents. Then he must be able to determine, 
for each of these types of performance, what rules to employ for generating a 
Job aid that will be appropriate for the class of problem he has at hand. In 
Order to generate a table of such information through research, it will be 
necessary to discover how to classify job-aid problems. And for each class, it 
Will be necessary to discover rules which can be employed to generate stimulus 
Materials that can be used as effective job aids. — f e: 

What is the proper goal for research that is designed to obtain infor- 
mation about classifying job-aid problems? At first thought, one might offer 

© answer that research should be directed toward describing the kinds of job 
tasks for which job-aid materials are required. Thus, one might attempt to 
'SCover useful ways of classifying job tasks so that the job-aid expert could 
Carn to classify any given task and then to enter a table by the type of task 
© find the rules for generating appropriate job-aid materials. Upon closer 
*Xamination. however, it can be seen that there is not a constant relationship 
"tween job tasks and the characteristics which job-aid materials must exhibit. 
act, it is rather simple to demonstrate that this relationship must be a 
unction of other factors. Consider almost any job task for which a job aid 
Might be constructed. Even though the performance is held constant, the 
“ture of the job aid must change as the initial repertory of capability of the 
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technician changes. Thus, for a group of technicians who can do A, B, and C, 
a job aid of one kind is required; whereas for a group of technicians who can 
do only A and B, a somewhat different job aid will be needed to foster the 
same end performance. 

If one cannot hope to discover what kind of job aid to build simply by 
considering the final performance that must be fostered by the job aid, where 
then should he look for a clue? Certainly not to the component performances 
which are exhibited by the technician before the job aid is introduced. The 
performance capabilities which the job aid must build upon are necessarily 
already parts of the technician's repertory, and the stimulus conditions which 
will elicit each one will have been determined by his past learning experience. 


Classes of Performance Change. If the classification problem is not that 
of classifying either the initial performance capabilities of the technician or 
the final job performance that must be exhibited, how indeed can the ques- 
tion of what to classify be answered? One answer is to say that it is the change 
between what the technician can do without the job aid and what he must be 
able to do with it that is important. If this point of view is taken, the research 
question becomes, “Of what classes of change in performance are humans 
capable, and what kinds of stimulus conditions are appropriate for fostering 
each kind of change?” 

The nature of the “change” concept can perhaps better be seen if the 
similarity between the job-aid situation and the training situation is considered. 
Figure 8.5 is a schematic description of a training situation in the same terms 
as those used in Figure 8.4 to describe the stimulus role of job aids. 

In this figure, S, and R, have the same meaning as in the previous figure. 
The figure shows that at Time 1 the learner will not respond with A, when Sz 
is presented, but that at Time 1 there are certain capabilities in his repertory- 
Further, the diagram shows that, if S, is presented at Time 2 along with 
certain training conditions such that R, is elicited, and if this continues either 
with varying or fixed training conditions until Time 3, then it will be possible 
to present S, and obtain R, without any supporting conditions. It should be 
noted that the initial conditions in these two paradigms (Time 2 and Time 3) 
are the same, and so are the effective final conditions. The final conditions do, 
of course, differ in that when the job-aid method is used, the job aid remains 
as part of the final picture, whereas when the training method is used, all 
supporting conditions are withdrawn. The most important similarity between 
these two paradigms, however, is the similarity between Time 2 of the job-aid 
paradigm and Time 2 for the training paradigm. In both cases, at Time 2 the 
final response is elicited by means of supporting stimulus conditions. 

This similarity at Time 2 is worth a good deal of discussion, because it 
provides a clue as to what must be done in research to develop information 
about how to construct job aids-as stimulus materials, and because it suggests 
a way in which job-aid research can complement training research. Let us 
consider Time 2 in a training situation. 
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TIME 1 


TIME 2 


Training 
conditions 


S. R, 


Figure 8.5 Schematic description of performance before, during, and 


after training. 


In setting up the training conditions for Time 2, the training expert 
should accept responsibility for accomplishing the objective as quickly as 
Possible, This means that he must, if he can, set up training conditions which 


Will make it highly likely that the learner will make a correct response on the 
first training trial. It must also be observed, however, that although this might 
* desirable, it is not necessary in a training situation. If the training expert 
Cannot think of training conditions which will work reliably on the first trial, 
© can fall back on a less effective procedure; that is, he can attempt to shape 
the desired behavior by means of rewards and punishments over a number of 
raining trials. This throws the burden of figuring out the learning problem 
Upon the learner rather than upon the training expert, and it is therefore apt 
tO be much less efficient than a training situation which has been designed in 
Such a way that the training conditions promote the acquisition of the desired 
Stimulus-response relationships. However, it is a permissible way of solving 

© problem in the training situation, and it is one which one frequently falls 
ack upon when he does not know how to set up optimal training conditions. 
Perhaps it is because it is possible to fall back upon a technique of 


Shin: n : Faut 
haping behavior by means of rewards and punishments that little training 
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research has been concerned with the question of performance functions of 
the type described in Chapter 2; that is, since we can fall back on rewards and 
punishments to effect training, we have not been motivated to explore another 
approach. It is important to note, however, that some of the functions dc- 
scribed in Chapter 2 are functions which are subject to change. Thus, while 
sensing is probably not subject to change, the functions of identifying and 
interpreting almost certainly are subject to change through learning. For 
example, if we present a Morse code signal at a high rate of speed to a naive 
subject, the sensing function operates in the same manner as in a sophisticated 
subject. On the other hand, the naive subject will have difficulty in reporting 
whether or not confusable Morse code signals are the same or different; 
whereas the sophisticated subject has learned to make these identifications. 
That is, some change has taken place in the identifying function as it is used 
by the sophisticated subject from its use by the naive subject. If we consider 
those human functions which are subject to modification or learning, we can 
see that the training conditions which must be employed to effect a change in 
one kind of function are frequently different from the training conditions 
which would be employed for another. When it is desired to set up condi- 
tions to attempt to foster correct responding on the first trial of a training 
experience which involves a modification of identifying, those conditions will 
be quite different from the conditions that will be set up to foster memorizing 
of a sequential routine. When we attempt to control the learning experience by 
means of training conditions, it is apparent that the conditions selected must be 
tailored for the kind of function we are attempting to modify. It follows that a 
profitable line of inquiry in training research is to attempt to identify the 
different kinds of performance functions which are subject to change, and then 


to identify for each kind of function a type of training condition which can 
be used to promote effective learning. 


Achieving Stimulus Control 


In developing a job aid, the designer must develop completely reliable 
stimulus control. While this might be considered a potentially profitable line 
of research in training, it is almost a necessary approach in job-aids research. 
One cannot fall back on the alternative of shaping behavior by means of 
rewards, punishments, and corrective procedures over many trials. He must 
set up stimulus conditions which will with certainty elicit desired responses on 
the very first trial. The job-aids designer is in a position similar to that of the 
training designer who must be successful on the first trial. For similar reasons, 
then, the job-aids designer must be able to identify the kind of performance 
change which he is attempting to effect, and he must be able to specify the 
type of stimulus control that will work for that kind of performance change. 
Job-aids research must thereforc be directed toward the identification of the 
different kinds of performance change which can be effected by stimulus con- 
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trol and toward the identification of the type of stimulus conditions which will 
be effective for each type of change. Because of the similarity between the job- 
aid situation and the training situation at Time 2, job-aids research of this type 
will almost certainly turn out to be useful to the training designer. It seems 
that the important feature of the job-aid situation is that it forces research to 
be concerned with the development of stimulus control, whereas this is not a 
necessary approach to training research—although it is certainly a desirable one. 

There has been virtually no job-aid research specifically focused on the 
development of methods of stimulus control for specific types of performance 
change. Such research could well start with consideration of what has been 
accomplished in training research to control response during training. In turn, 
it can be expected that job-aids research focused on this problem will turn 
Televant findings back to the area of training research. 

For the most part, the job aids now in use are employed to effect a 
Change in performance which is comparable to memorization of a sequence. 
(See Rees, 1959: Rees and Copeland, 1959.) Most job aids substitute for the 
memory function by eliciting comparable performance by means of stimulus 
Control. In order to round out discussion of job aids as instruments of stimulus 
Control which must be tailored to the kind of function being controlled, let us 
Consider another example of a requirement for a job aid which is concerned 
With identifying. Figure 8 6 presents two PPI oscilloscope patterns which must 
be employed in troubleshooting a certain electronic equipment. if the tech- 
Nician sees Pattern A, he knows that the malfunction is in Unit A of the 


PATTERN A PATTERN B PATTERN C 


atterns used in a job aid which require special 
“identification.” 


. Figure 8.6 Three p 
lob aid conditions to promote reliable 


SQipment. it he sees Pattern B, he knows that the malfunction is in Unit B of 
Mhe equipment, The problem to be solved is how to present this information in 
s à way that the technician will always respond correctly by identifying 
™ A when he sees it on the PPI, and so that he will always correctly 


ify p ses it on the PPI. The problem is complicated 
attern B when he sces r from time to time on the PPI. It is 


hown on the job aid in the same 
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way that they are shown in the figure, the technician will frequently confuse 
them. Certainly, if one were to approach this as a training problem, he would 
expect A and B to be confused during training, and that during early trials the 
learner would call Pattern A by the name B and the Pattern B by the name A 
a fairly high proportion of the time. The same condition occurs in the job-aid 
situation when the information is presented as shown in Figure 8.6. This looks 
like a rather simple sensing problem when Patterns A and B are scen side by 
side, and one might therefore wonder how a technician could ever confuse 
them. The fact that they are confused may be explained by noting that each 
is a triple-criterion stimulus. Thus, Pattern A must be a spiral, it must be 
segmented, and its segments must alternate; whereas, Pattern B must be a 
spiral, must also be segmented, but the segments must be parallel to each 
other. Hopefully, the problem of presenting this information on a job aid can 
be solved in such a way that the technician can rapidly and reliably identify 
Pattern A when he sees it on his PPI and also identify Pattern B. Such a 
solution will perhaps also make possible the specification of a stimulus-control 
technique which can be extremely useful in a training situation requiring the 
same kind of change in performance capability. 
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THE PARTICULAR CAPABILITIES NEEDED BY 
men to perform the variety of tasks required in system 
operation are established by a process of training. The 
selection of men possessing certain fundamental abili- 
ties, and the classification of people into ability cate- 
gories, primarily serve the purpose of establishing a 
desirable starting point for training, as a previous 
chapter has indicated. Following this classifying procedure, measures must be 
taken to alter the “raw” human capabilities, to mold them into the particular 
state of competence which will insure the smooth and efficient meshing of 
the man-machine combination when in operation. The procedures used to 
bring about this increased degree of capability in man, resulting in heightened 
and increasingly skilled performance, are what are referred to as training. 
Several different subordinate purposes of training may be distinguished, cach 
having its greatest relevance at different stages of system development. The 
following chapter is primarily concerned with individual training, which is 
ideally designed to establish in the individual man the skills he requires to 
make effective use of a machine he will operate, or of tools he will use, 
within a particular subsystem. The varieties of team training and system 
training, carried out to establish and refine interactive procedures within the 
system, are discussed more fully in later chapters. 

The procedures of training are based upon a number of principles and 
findings of the science of psychology, as well as on a growing body of tech- 
nology. They are cast into practical form in several kinds of situations de- 
signed to foster human learning, including the individual practice session, 
the training device, the classroom, and on-the-job training. The effectiveness 
of these procedures is assessed, both during the training and afterward, by 
the use of tests of human achievement and measures of human performance 
within the system context. 

The conduct of training for systems often gives new emphases to funda- 
mental questions about human behavior which are of potentially great impor- 
tance to the scientific understanding of the process of human learning. Among 
these are the meaning of "knowledge" and "skill" and their relation to human 
performance; the nature of inferences which may legitimately be made regard- 
ing the factors which determine the learning of these entities. The ways in 
which motivation operates in learning is another matter that is highlighted by 
training programs. Finally, there is the problem of arranging the content and 
sequence of training sessions, which is the kind of problem the learning 
psychologist thinks of as the "programming" of learning. 
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CONCEPTS OF TRAINING 
Meredith P. Crawford 


The concepts presented in this chapter concern both the process of 
training and the application of techniques of research and development to 
that process. The discussion is limited to the training of individuals, since 
the training of groups is discussed in Chapter 11. Also excluded from con- 
Sideration are training devices, which constitute the subject of Chapter 10. 

For purposes of this discussion, training is defined as that process by 
Which individuals learn the knowledges, skills and attitudes, not previously 
' their repertories, which will fit them to function as human components in 
3 System, This training may take place in a school, before the individual is a 
member of the system, or on the job, after he has become a human component 
Within jt, Major attention will be devoted to formal training given before the 
Man enters the job, although some consideration will be given to on-the-job 
training. It is recognized, although not discussed in this chapter, that the indi- 
vidual will continue to modify his behavior even after he has reached a mini- 
Mum level of competence on the job. This improvement in performance is, 
M Part at least, a function of the individual’s learning to interact with other 

embers, and so is properly treated in Chapter 11. I 

The notion of a system will be employed in various ways. It may refer 
is à large organization, such as an industrial concern or one of the military 
E vices; to a subsystem within the larger organization, which performs some 
articular function within the larger system; or it may be limited to a particu- 
ies Thàn-machine or man-weapons combination. The idea will even be 
tio ended to the single individual in the treatment of his postres for and func- 
a, 8 à human component in a system. Since systems exist within systems, 
8 Particular system of interest may be regarded both as a sub- and a super- 

analysis. Such catholic use of the term, 
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which suffers from a lack of precision, is justified in order to gain the analytic 
advantage of attempting to specify, for each entity so labeled, its purpose 
and the nature of its input and its output. 

To illustrate the usefulness of thinking in terms of systems, as well as to 
suggest the point of view from which this chapter has been written, a dis- 
tinction will be suggested between the meanings of the terms training and 
education. While it is obvious that both are concerned with human learning, 
and the technical problems of content and method are shared by both, they 
differ in terms of purpose. Training is undertaken to serve the needs of a 
particular system, while education aims to fit persons to take their places in 
the many systems of society. In training, a program of instruction is designed 
to fit the trainee for a particular job within the system, while a specific job is 
usually not foreseen for the student in an educational program. Furthermore, 
the worth of the training program is to be estimated in terms of its contribu- 
tion to the output of a particular system. Education, on the other hand, is 
evaluated largely in terms of the growth of an individual. Thus, in general, 
training establishments are owned and operated by business corporations and 
military services, and students are paid to attend as employees of the parent 
organization. In contrast, educational institutions are owned privately or by 
the state, and tuition and other costs are borne largely by the individual oT 
by society in general. Therefore, the objectives, policies, and values of à 
parent system are incorporated in the training program. Such a program has 
its primary meaning and utility within the sphere of the parent system. It fol- 
lows that the parent-system managers would tend to expend no more On 
training programs than, in their judgment, is required to develop the knowl- 
edges, skills, and attitudes in new employees believed necessary for their 
functions as various kinds of human components in their system. 

In actual practice, many courses of instruction in training and edu- 
cational institutions are very similar. This is particularly true at “higher-level” 
schools, such as the War Colleges of the military services, which are usually 
considered educational establishments. The reason is that they train (or edu- 
cate) generalists, men who can function at the apex of various systems, 
understanding the relations between several systems, and who can bring 
innovations into the operation of their particular systems. Similarly, business 
and governmental organizations sometimes send employees to universities for 
courses of a general nature, the exposure to which is believed to be in the 
best interest of the parent system. 

It is apparent that the training process about which this chapter is con- 
cerned is a creature of the system it serves. Much of what will be said about 
the nature of training and the methods of research and development appro- 
priate for it can be deduced from this fundamental proposition. 

The remainder of the chapter is divided into three principal parts. Begin- 
ning with an identification of types of systems toward which the training of 
individuals may be directed, the first section includes an account of the sub- 
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Systems of a large organization which are concerned with personnel and 
training. In the second section, attention will be turned to research and devel- 
opment as applied to specific training programs, illustrated by an example 
from Army training rescarch. The final section considers the central problem 
of training research, the management of the learning process, and the bearing 
of psychological science on this problem. 

It will become apparent that this chapter is derived from experience in 
military training. Most of the illustrations will be taken from studies performed 
for the United States Army. While it is believed that many of the observations 
Which will be made apply also to industrial training, particularly for defense 
industries, a relatively small amount of direct attention will be given to train- 
Ing in commercial establishments. 


SYSTEMS IN RELATION TO TRAINING 


a 


The Parent System 


. For a large organization, such as one of the military services or an exten- 
Sive civilian enterprise, the term parent system is a convenient designation. 
Such an organization may be regarded as a system in terms of its single gen- 
eral purpose or its collection of interrelated purposes and in terms of its 
identifiable inputs and outputs. Some large organizations may be systemic 
1n nature only with respect to a certain general purpose or as a function of 
Some common interaction, such as the movement of money for the purpose 
9f earning profit for the entire, but otherwise loosely federated, organization. 
Phe term parent suggests the creation of several entities within the over-all 
System, which contribute in various ways to the major purpose of the total 
System and are responsible to it for direction. The management group or 
headquarters organization of the parent system may also be regarded as a 
Subsystem, whose purpose is to control the entire enterprise. This management 
group may even be regarded, for analytic purposes, as an extension of that 
Single biological system represented in the person of the organization’s chief 
executive, 

Certain subsystems perform operations contributing directly to the major 
Boals of the parent, yielding outputs which provide components of, or con- 
Stitute without modification, the outputs of the parent system. These will be 
Called operational subsystems; the various major units of a military service, 
the Separate production plants of a large manufacturing concern, and the 
Fegional offices of a sales and service firm are illustrations of operational sub- 
qims, Other subsystems provide support for some or all of the operational 
Bo Systems, among which are those that furnish and maintain the human 

™ponents: the personnel subsystem and the training subsystem. The latter 
ay be a part of the former subsystem or may be differentiated from it. 
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Types of Operating Subsystems 


The kinds of functions required of the human components of operating 
systems are the primary determinants of training. As indicated in Chapter 2, 
information-processing is taken to be one of the principal functions of the 
human component, while functions involving the application of power, per- 
formance of heavy work, or transportation are largely relegated to machine 
components. However, for consideration of training, it may be worthwhile to 
suggest a range of man-machine systems which require human functions ex- 
tending from heavy manual labor to the most sedentary forms of data proc- 
essing or decision making. A useful continuum in terms of the relative amount 
of hardware critical to system function has been defined between extremes 
that have been labeled “man-ascendant” and “machine-ascendant” systems.’ 

An extreme example of the man-ascendant system is a football team. 
The only absolutely necessary piece of “hardware” is the football itself— 
many sandlot games are played without the appurtenances of goal posts, 
chalk-lined playing field, or ten-yard chain. The players themselves supply 
the power, the movement, the striking force, as well as the information 
processing, the memory and the tactical decision making. Hence, training for 
football involves increasing physical stamina, strength, and speed, along with 
the development of psychomotor skills such as blocking, tackling, and ball 
handling, as well as the more symbolic or intellectual signal memory, percep- 
tual judgment, and planning. 

The machine-ascendant system, on the other hand, is well illustrated by 
a guided-missile battery, where the power output of the system is infinitely 
greater than the combined muscular output of all the men in the battery, and 
in which even the erection of the missile for launching is powered by the 
nonhuman devices. The major function of the personnel is in interpreting: 
coding, troubleshooting, information processing, and decision making. 

A third example, the typical infantry unit, seems to occupy a position 
somewhere between these two extremes. Weapons supply a destructive power 
output much greater than the human muscles can provide, yet physical 
strength and stamina are essential for walking, running, and climbing, and 
for providing support for hand weapons. Symbolic behavior in coordination 
and decision making is, of course, also required. 

A distinction which often parallels the contrast between the man- 
ascendant and machine-ascendant systems is particularly interesting from the 
training point of view. The man-ascendant system characteristically takes its 
physical form, its spatial configuration, from the men themselves—the infantry 
squad moving in diamond formation is, as a whole, a physical weapon, flexi- 
ble, yet dependent for fire and maneuver on its several “Gestalts.” In the 
machine-ascendant system, as the bombardment aircraft, the naval combat 
ship, or the complex newspaper printing press, the machine itself defines the 


1 Follettie, J. F., personal communication, 1959. 
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Physical position and lays obvious constraints upon the performance of the 
men. Differences in the extent to which the machine defines the function of 
the crew result in different requirements for individual skills and modes of 
communication, as well as for different techniques of control and leadership. 


The Personnel Subsystem 


The functions of recruiting, selecting, classifying, training, upgrading, 
Promoting, and counseling members of the parent system may be accom- 
plished for the parent system as a whole or for some large operating sub- 
System by a personnel subsystem, whose operation reflects the policies of the 
Parent system on the management of people. Insofar as there is a differen- 
tiated Personnel organization, it may be regarded as a subsystem whose pur- 
Pose is to furnish competent human components for operating units at the 
tight Place at the right time. This competence is in part determined by the 
nature and effectiveness of the training subsystem which, if not a component 
of the personnel subsystem, is obviously critical to its function of personnel 
Supply, 

Personnel policy and the functions of the personnel subsystem are of 
prime consequence to training. The average and range of general and special 
aptitudes, prior knowledges and skills, and attitudes of students furnished for 
training are among the determinants of the content and mode of instruction. 
The efficiency of the distribution of graduates of training to jobs in operating 
Subsystems for which they are trained contributes to the effectiveness of the 
Operational support rendered by the training subsystem. Finally, policies on 
rotation of personnel for upgrading and career management affect the extent 
to which training programs will be specific to particular jobs or will offer 
once training for a hierarchy of positions of increasing breadth and respon- 
Sibility, 


The Training Subsystem 


Certain general features of typical training subsystems are of conse- 
Wence to the psychologists who intend to make a contribution in this field 
research and development. These concern the training subsystem as a 
ole and its relation to the parent system. Many characteristics will have 
O be taken as constraints in the planning for new training; others are subject 
9 Some alteration on the basis of research information. ÁN 
T€ The training subsystem consists of the entire group of courses, facilities, 
ion Personnel devoted to training by the parent system. One of the largest 
ülning Subsystems ever invented served the Army Ground Forces during 
dae War II. An instructive account of the interlocking operation of several 
is erent training schools and facilities for the. formation of an Army division 
Presented in a volume by the Historical Division of the Department of the 
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Army (Palmer, Wiley, and Keast, 1948). This account of the determination 
of appropriate schedules, quotas, and attrition rates for the system as a whole, 
as well as the content of individual courses, indicates the extent and com- 
plexities of a large training subsystem. While research and development have 
been performed mainly on particular training programs within the subsystems, 
as will be discussed in subsequent sections, it will be profitable at the outset 
to view the training subsystem as a whole. This will be accomplished under 
the common systemic rubrics of purpose, input, and output, together with a 
listing of other important characteristics. 


Purposes of the Training Subsystem. Within the general purpose of 
providing individuals with requisite knowledges, skills, and attitudes which 
enable them to take their places in the operational systems of the parent 
organization, more specific objectives are incorporated. To highlight some of 
these, one may begin by asking why a formal training system should exist at 
all. Why not apprentice new people directly to older workers already on 
the job? 

Apprentice training works well enough if certain conditions prevail in 
the operating subsystem. It must be stable, operating in its normal mode, and 
experiencing little time pressure for bringing new employees up to minimum 
proficiency. In addition, the jobs must be relatively simple and safe. Neither 
modern industry nor a military establishment operates under these conditions. 
Jobs have become increasingly complex, especially in their demands for 
information processing and symbolic manipulation. In addition, the possibili- 
ties for damage to the person or equipment from incompetent handling have 
become greater. Furthermore, there is constant change in military and indus- 
trial job requirements, which call for quick regrouping of skills and knowl- 
edges among operators and maintenance personnel. Finally, and perhaps most 
important of all, separate training subsystems are required to train people for 
jobs in new operational subsystems which are still under development, SO 
that they may function as soon as the equipment is ready. Therefore, a sub- 
system for training exists which keeps employees away from the operational 
scene until they have acquired a certain minimum of knowledges and skills. 
Training may then be completed on the job in an operating subsystem. 

The training operation must provide for a uniform performance in a paf- 
ticular kind of job as it may exist in several operating subsystems of the parent 
system. A man trained to the specifications of a particular occupational 
specialty must be able to fill exactly the requirements of that human compo- 
nent position. For better or for worse, military personnel systems have 
adopted the hardware production philosophy of “the interchangeability of 
parts, mass produced." When, in a large training subsystem, individual train- 
ing programs for the same job are offered at different locations by different 
instructors, controls must be exercised to insure uniformity of the trained 
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product. The development and exercise of these controls in terms of pro- 
ficiency assessment constitutes an important administrative and research 
problem. 

The provision for innovation in operations is a purpose also served by 
the training subsystem. For example, men must be prepared for jobs which 
do not yet exist. To provide the means for accomplishing this training, 
Planners need to team up with hardware planners and equipment developers 
to share advance information required for building new training programs. 
It is not always easy to specify the types and sources of this kind of informa- 
tion. To facilitate this exchange, there is embedded within the United States 
Army's over-all training subsystem the Combat Developments System, whose 
function is to look ahead toward new doctrine, weapons, and tactics, and to 
anticipate training requirements. Innovation may also be provided by the 
training subsystem even when no new major concepts or weapons are in- 
Volved. Often new operational concepts and new procedures are devised 
Within a training subsystem when training developers carefully examine the 
Jobs for which they are constructing training. In this way, the industrial or 
military school system may furnish something of the same novel approach 
to operational system practices that is provided by the universities for our 
Culture as a whole. 

In response to requirements of the personnel subsystem, training fulfills 
another purpose in providing for the upgrading of personnel and the storage 
of knowledges and skills within them. This practice has important benefits 
to the individual, in adding to his knowledge, broadening his systemic out- 
look, and increasing his self-esteem. Training is equally important to the 
Parent system in that it can provide a supply of potential managers and 
leaders who may be called upon for additional duty and responsibility in the 
future, This is especially true in the military services, where advanced work 
In higher-level schools prepares field-grade student officers for assumption of 
Tesponsibilities commensurate with higher rank in the event of rapid expansion 
Ih time of emergency. Incidentally, it may be observed that training research 
9n such programs suffers from one serious handicap: since the graduates of 
Such schools often do not immediately perform the duties for which they are 
trained, it is difficult to measure the effectiveness of the training programs. 

E It is apparent that these several purposes of the training subsystem may 
all be subsumed under the major purposes of the operational subsystems and 
9f the parent organization to which they belong. It is therefore clear that the 
Ultimate test, for example, of military-training effectiveness is in the contri- 
“tion it makes to performance of units on the battlefield. However, to 
“Tret out this contribution from the many other sources of variance within 
© total system is extremely difficult. Similarly, the quarterly earnings report 
Xo large corporation should reflect, in some measure, the effectiveness of 
company training programs. While several articles on the costs of indus- 
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trial training have appeared in The Wall Street Journal, the writer has not 
yet seen a financial statement which attributes a change in company profits 
to a change in the proficiency of personnel brought about through training. 


Input to the Training Subsystem. Members of the parent system, or 
candidates for membership, with various sets of knowledges, skills, and 
attitudes within their repertories constitute the input to training. The content 
of their repertories must be assessed very carefully and completely if training 
is to be maximally effective, which is the purpose of selection and classifica- 
tion procedures (see Chapter 7). Because of limitations in available man- 
power, the possession by possible candidates of incorrect or inadequate infor- 
mation about available job opportunities, and imperfect selection procedures, 
the personnel system may work in less than ideal fashion. Heterogeneity of 
input places particular burdens on a training system which has a rigid output 
quota to meet on a fixed schedule. Adaptation of training to varieties of apti- 
tude and background characteristics of trainees constitutes a major research 
problem. 


Output of the Training Subsystem. Individuals who can perform the 
functions of a human component in an operating subsystem constitute the 
output of school or on-the-job training. The minimum level of performance 
required of particular human components to produce an acceptable output 
from an operational subsystem is usually quite difficult to specify. While 
inadequate proficiency of graduates is the more common shortcoming of 
training subsystems, there are probably several instances of unnecessary over- 
learning of certain knowledges and skills for a particular job. Such additional 
capabilities of individuals may be of some use for career advancement tO 
other positions, but their acquisition during a particular job-oriented training 
program should be evaluated in terms of training costs weighed against ad- 
vantages gained by the parent system in subsequent employment of the indi- 
vidual. Smith (1960) has offered a system model for determining that level 
of proficiency of training graduates which should strike an economic balance 
among the costs of the training subsystem, the immediate requirements of an 
operational subsystem, and the long-term needs of the parent system. A 
judicious combination of school and on-the-job training may help in solving 
the equation. It should be added that learning continues to take place after 
the man becomes a component in the operational system. As will be shown in 
Chapter 11, teams continue to improve their total performance as their mem- 
bers learn to work together. Through group discussion and interaction, new 
and more effective ways of performing are invented in the operational 
situation. 


Other Characteristics of the Training Subsystem. In addition to these 
formal subsystem specifications, other kinds of characteristics are worth noting 
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by the training researcher. He needs to find out about the general policy of 
the parent system with respect to the training subsystem. Does management 
consider training important, or does it put up with it as a necessary but 
unproductive expense? Does it grant status to training personnel? Most impor- 
tant, perhaps, does it look to the training subsystem to provide an important 
ingredient in the total system operation? During the past few decades, there 
has been a marked increase in the positive attitude of management toward 
training, which has ranged from shop-level training for beginning employees 
lo management courses for potential executives. This interest perhaps reflects 
à general increase in public interest in education. 

Another general point of concern is the working relations between train- 
ing and operational personnel. In some situations, operational personnel have 
à tour of duty in the training establishment, while in other cases, training 
Personnel have a career of teaching and school administration. Presumably, 
the former practice brings constant operational emphasis to the training pro- 
gram, while the latter provides for conservation of teaching and training- 
management skills. The crucial question concerns the extent of communica- 
tion between members of the operational subsystems and the training 
Subsystems, The importance of feedback of information about operations to 
the training subsystem can hardly be overemphasized. Whether this commu- 
Nication is best done by having training personnel make frequent contact 
With operators or by rotating operators as instructors is open to debate, but 
Neither practice relieves training researchers from the responsibility for gain- 
Ing a thorough knowledge of operations. . 

. Finally, the training researcher must assess the boundaries of the train- 
ing subsystem. He must form some idea of the likely funds, space, locations, 
and terrain which are currently devoted to, or may be potentially used for, 
‘raining purposes. This kind of information provides a background for the 
Necessary assessment of costs and gains which will have to be made before 
a decision can be reached by parent-system managers on the adoption of a 
New training program. Research personnel should be able to contribute to 


this kind of assessment. 


© RESEARCH AND DEVELOPMENT FOR 
SPECIFIC TRAINING PROGRAMS 


d development discussed in the remainder 

his chapter are directed at the construction Or modification of specific 

Courses of instruction, rather than at the management of entire school pro- 

Srams or the training subsystem as a whole. In this section, a discussion of 

; * different contributions made by research and by development to the 
Provement of training will be followed by.the presentation and illustration 
à Particular sequential method for developing training. 


of The methods for research an 
t 
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Research and Development in the Field of Training? 


Methods for the planning and conduct of educational and training 
courses have a long history (see, for example, Brubacher, 1947). These 
methods have come to constitute something of an art. The kinds of informa- 
tion and skills which support it come from many sources, of which experi- 
mental psychology is only one. While the psychologist has the potential for 
contributing to this art through research on learning and behavior measure- 
ment, he has to add a demonstrable increment to its effectiveness to make 
his contribution felt. This contribution comes about through participation in 
development, psychotechnology, and basic and applied research. 


Development. The process of working with an idea or model or design 
until it is modified into an operable, maintainable, and salable item, usually 
capable of being mass-produced, is known as development. Often this is a 
long and difficult undertaking. Potable, fresh water can be realized from sea 
water by various processes of distillation or freezing; however, the determina- 
tion of which process can do it cheaply and reliably enough for practical use 
remains a developmental enterprise. Often it is not theory but a vast amount 
of specific technical information and practical know-how that is utilized in 
the developmental process. Research may result in the basic idea for a new 
machine or product, but as a research result, it cannot be put to a practical 
use. Only after developmental steps are taken can the idea be of any practical 
significance to the general public. 

A specific training program with its associated lesson plans, textbooks, 
manuals, training aids, and supplementary materials has to be selected and 
assembled, tried out, and put together to build a workable course. In à 
university course at the graduate level, this is often done on an ad hoc, non- 
standardized basis by an experienced professor whose subject is unique in 
the university's curriculum and whose students are highly motivated toward 
self-help. However, a training program intended for large numbers of people; 
taught in several places by many different instructors, must be put together 
in advance into a carefully engineered, well-documented, and completely sup- 
ported “training package," which can be effectively administered, if necessary, 


by persons relatively unskilled in the teaching arts. This package comes about 
through a process of development. 


Psychotechnology. The ideas which the developer of training programs 
uses may come from sources as diverse as papers on learning theory and the 
accumulated wisdom of an old drill sergeant. Insofar as the ideas are tested 
and predictable consequences are shown to follow the same antecedent con- 
ditions, a body of information is accumulated. Such a body of information 


2 The ideas presented in this section have been developed during discussion with 


Dr. John L. Finan, to whom the writer is greatly indebted. They are elaborated in 
greater generality in Chapter 14. 
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becomes highly useful in the limited field from which it is derived, and it may 
be called a technology. It is composed of a set of low-order laws which apply 
to the limited universe from which they were induced. The laws are not 
explanatory; they do not utilize constructed concepts to mediate between 
antecedent and consequent conditions. Instead, they are simply descriptive 
and useful in the context of the "if-then" relationship. 

A technology of training is thereby in the process of gradual formation 
and is taking its place among other psychotechnologies relevant to the man- 
agement of human behavior. The chief areas of this branch of psychotech- 
nology will be specified in the exposition of steps in training development in 
à subsequent section. It may be noted, by way of comparison, that the psycho- 
technology of aptitude testing and performance measurement is much further 
advanced than that of training, reflecting the concerted effort of psychologists 
in these fields, particularly during and since World War II. 


Basic and Applied Research. In the strictest sense of the word, basic 
research is undertaken from a theoretical point of view with the aim of 
testing hypotheses. The biological organism chosen for study and the methods 
€mployed are selected in terms of their appropriateness to the conceptualiza- 
tion under examination. The results of such research have wide generality 
along specified conceptual dimensions. The results from theoretical-learning 
Tesearch are a source of ideas for use in the psychotechnology of training. 
When an idea appears to be relevant to a particular problem, it is tested under 
the conditions of application; this is an activity within the province of applied 
Psychology. It may be noted that research in the psychotechnology of train- 
ing only rarely affords an opportunity to test for conceptual generality in the 
Manner appropriate to basic research. The psychologist working in the prac- 
Ucal field of training needs to keep well in mind the distinctions between basic 
and applied research, on the one hand, and psychotechnology and develop- 
Ment, on the other. Otherwise, he may be frustrated in a misguided effort to 


accomplish both within the same investigation. 


The Goals of Training Development 


It is apparent from the above discussion that the goals of training devel- 
OPment differ from those of psychotechnology or basic or applied research. In 
evelopment, certain kinds of ends may be sought, depending on the needs 
of the parent system. Among the important objectives of training, three kinds 
9f goals may be cited, not necessarily mutually exclusive. They are: increased 
Proficiency, decreased training time and costs, and decreased aptitude require- 
ents, 


Increased Proficiency. Probably low proficiency of training graduates 
the most common operational problem in both military and industrial 


t ; s Res 
taining, It is often not easy to state precisely how deficient such graduates 


is 
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are, because it is difficult to specify in quantitative terms the proficiency 
requirements for each job within the operational subsystem. A performance 
test can often be used to assess the proficiency levels of persons holding vari- 
ous jobs within the operational subsystem, since it isolates the individual from 
the system and supplies, in a simulated fashion, the systemic inputs to the job. 
For example, in order to establish reasonable goals for a training program for 
Army radar technicians, a proficiency test was administered to men who had 
been on the job at air-defense sites around the country for varying periods of 
time (Baldwin et al., 1957). Figure 9.1 illustrates the average performance 
of these technicians plotted against average months of experience. The base 
line of 100 is the average performance of training-school graduates, and the 
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Average proficiency-test performance by months of field 
experience. 


(Adapted from Baldwin et al., 1957, by permission of the 
Human Resources Research Office, The George Washington University.) 


standard deviation was 20. The dotted horizontal line represents an estimate 
made by system managers of the minimum necessary level of proficiency- 
This estimate, while useful, was only a general judgment, rather than a pre- 
cise, quantitative derivation from measured interactions within the opera- 
tional subsystem. Since this level was not achieved until after men had been 
on the job for some time, the goals for a training program could be meaning- 
fully set in terms of that level of performance achieved by experienced 
technicians after several months of on-the-job experience. 

In the development of training for new jobs in new systems, it is neces- 
sary to specify in advance some level or range of performance which will be 
adequate for the needs of the system. The evaluation of the training program 
developed is then absolute, rather than relative, and training programs will be 
continuously engineered or developed until performance of trainees falls 
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within tolerable limits. The specification of human performance, or the ranges 
through which job output may vary to meet the requirements of the system, 
serves a purpose analogous to the specification of component performance 
standards and reliability of hardware components in the operational sub- 
system. (For a fuller discussion, see Chapter 12.) 


Decreased Time and Costs. Since time spent within the training sub- 
system is essentially nonproductive with respect to the operational subsystem, 
it is usually important to reduce it. In industrial establishments, training 
time can be judged in terms of dollar costs—the longer, the more costly. In 
military systems, time is of the essence during a period of national emer- 
Bency, when every day counts in readying a trained military unit for opera- 
tional missions. Insofar as it is possible to divide the total training required 
between school and job, and accomplish a substantial part of it on the job, 
the new man may have a least limited utility in the parent system fairly early 
in his employment. 

In addition to the personnel costs of training, involving the pay of stu- 
dents, faculty, and support people, which increases with the length of training, 
much expense may be involved in the buildings, outdoor training areas, and 
the operational or simulated equipment involved in training. An analysis of 
the specific knowledges and skills to be learned, and an inquiry into the 
simplest way of presenting training stimuli and providing for practice may 
Teduce the expense of training. For example, an experiment ( Denenberg, 
1954) on the amount of learning achieved on a $27 wooden mock-up and a 
$10,000 realistic simulator revealed an equal amount of learning. Training 
devices and simulators, even though expensive, may be cheaper and more 
feasible than practice on certain operational equipment, as for example, cer- 


tai ae 
ain modern missile systems. 


. . Decreased Aptitude Requirements. When the manpower supply is 
limited and trainees with high aptitudes for certain kinds of training are in 
Short supply, an important objective of the training research and development 
fort may be to develop the content and method of instruction, or both, so 
that men of relatively low aptitude can learn to do the job, thus increasing 
the number of potential candidates. In some cases, this may be achieved if 
the training content is limited strictly to that which is relevant to a specific 
Job, and no attempt is made to supply any underlying theory or more general 
struction which might be useful to trainees of higher aptitude in fitting 
em for rapid advancement to positions of greater responsibility in accord- 
ance with some career pattern. Here, personnel policy and the operation of 
with Sonnel subsystem may have a very important influence on actions 
te In the training subsystem. An illustration will be given later in this chap- 
T of an instructional method designed to obtain this kind of result. 
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Major Steps in a Training Development Project 


During the past several years of work in military-training research and 
development, the principal activities required in an orderly development of 
a training program have become apparent. A sequence of these activities is 
presented in diagrammatic form as Figure 9.2. While the seven steps indi- 
cated in the seven boxes represent somewhat arbitrary divisions of the work, 


Analysis of the military system from the human factors point of view 


I 


Analysis of the particular job 


Specification of knowledges and skilts] 


Į 


Determination of training objectives ] 


[ 


Construction of the training program | 


o programming of instruction 
e practice materials 
9 achievement testing 


\ 


| Evaluation of the training progam | 


Development of measures 


Figure 9.2 Steps in the development of training. 


effort of the kinds indicated at each step must be expended in order to accom- 
plish a complete job of building appropriate training and of testing the 
result, In the main, the work is dependent on an underlying psychotechnology. 
various aspects of which are treated in this and other chapters. When the 
psychologist is working at the development of a specific training program. 
he is functioning as an engineer, bringing his technological information and 
skill to bear on each step. In such work, he is able to see the utility and the 
limitations of the current technology, and so becomes well oriented toward 
further research designed to strengthen the technology. In succeeding para- 
graphs, each of the seven steps will be identified and reference will be made 
to chapters in this book where the underlying technology of each is discussed. 
In the next section, the developmental sequence will be illustrated by an 
account of the development of a military training program. 


Analysis of the Operational Subsystem. An analysis of the operating 
subsystem is the necessary starting point to determine the characteristics 
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required of both the hardware and human components of the system. The 
human-factors aspect of the analysis is the basis for job allocation and speci- 
fication, and from this aspect it is possible to determine the relative contri- 
butions to human component performance which may be made by selection 
and classification of individuals in accordance with the knowledges, skills, 
aptitudes, and attitudes they already possess; by human engincering in adapt- 
ing displays and controls for optimum linkages between human and hardware 
components; and by training. The balance of effort among these three kinds 
of Psychological work can be determined during the system analysis. In the 
Case of a particular system and a certain available supply of manpower, the 
Solution may involve much more effort along one of the three lines than along 
the other two. In addition to analyzing the operational subsystem to find the 
functional characteristics required of each human component, it is also in this 
Step that personnel policies of the parent system are examined to determine 
whether people may be expected to move about within this subsystem from 
job to job or may be expected to leave it for other jobs in other subsystems. 
Such information has an important bearing on both selection and training 


Procedures, as indicated in Chapter 7. 


Analysis of the Particular Job. Next, out of the analysis of the whole 
System, the particular job is studied to determine the inputs to the job from 
the rest of the system and the required outputs. The type of job analysis 
e—one that is useful to the personnel subsystem, 


Which may also occur her 
and promotion—is of secondary 


volving classification for purposes of pay 
Concern to the training developer. 
At this point in the developmental sequence, a division of labor is intro- 
duced. On the left-hand side of Figure 9.2 are shown the steps involved in 
developing the curriculum of the training program itself, while the right- 
and side pertains to the measurement of job proficiency. These efforts come 


together in the evaluation step at the bottom. 


. Specification of Knowledges and Skills. Within the parameters of re- 
quired inputs to and outputs from the job, the processes by which the indi- 


Vidual makes these transformations are identified. These are psychological in 


Nature—sensing, discriminating, remembering, deciding and choosing, which 
and skills required for the 


are implied in the specification of the knowledges 
Job. Chapter 6 discusses procedures of job analysis and knowledge and skill 
SPecification, while a suggested set of categories for the processes involved 


in ample : 
transformation is given in Chapter 2. 


i Determination of Training Objectives. From the knowledge and skill 
Pecification, and from an assessment of the probable content of the reper- 
tories of persons to be trained, it is now possible to state in fairly precise terms 

* objectives of the training program as a whole. The goals may be limited 
9 the Specific job or may contain elements furnishing preparation for other 
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jobs later in a man's career. In terms of the specific job, it is at this point 
that decisions may be made about the optimum combination of school and 
on-the-job training, and on the kinds of aids, such as manuals and check 
lists, which will be made available for use on the job (see Chapter 6). 


Construction of the Training Program. This step involves the selection 
of specific subject matter to be used in the instruction, the programming of 
the material, and special techniques which may be designed to motivate the 
trainee. Opportunities for practice may be provided in the form of training 
devices and simulators, whose general characteristics are specified at this 
point. Included in this "package of training" are achievement tests for each 
part of the training program, to provide a measure of the student's progress 
and to maintain his motivation. They measure only parts of the total required 
job performance. The technology for building this kind of test is quite well 
known, perhaps because the older technology of education has necessarily 
been limited to achievement tests, rather than to the ultimate job-performance 
tests. The technology available for the support of this step, as well as the pre- 
ceding one, will be discussed in a subsequent section of this chapter as well 
as in Chapters 10 and 12. 


Development of Measures of Job Proficiency. As shown on the right- 
hand side of the figure, a proficiency test of the job as a whole is developed 
from the information about inputs to and outputs from the job determined 
in the job analysis. The test simulates these inputs yet isolates the man fron 
the operational subsystem, in order to measure his output without interfer- 
ence with the subsystem. Acceptable standards of performance on this test 
are to be derived from the analysis of the subsystem, which, ideally. should 
quantify minimum necessary outputs from all the jobs. The technology of the 
measurement of human performance is considered in Chapter 12. 


Evaluation of the Training Program. The proficiency test is used as an 
instrument in evaluating the training developed. The measurement may in- 


volve the comparison of equivalent groups of persons trained by one or more 
methods, as in the case of a new training. program's being proposed as 8 
substitute for an old one. If the operational requirement is to increase pro- 
ficiency, substantial gain in proficiency by the new training must be shown. 
while if shortened training time or decreased costs is the goal, it may only be 
necessary to demonstrate that the new program yields the same degree of 
proficiency as the old. This evaluation step serves two purposes: it supplies 
parent-system managers with information on which to base a decision for or 
against adoption of the training so constructed, and it contributes quantitative 
data to the store of technological information. Some of the technical problems 
in this kind of measurement, especially as they bear on the choice of proper 
research design for psychotechnology, as contrasted with basic research, are 
covered in Chapter 14. 
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An Account of a Military-Training Development Project 


To illustrate the research and development process outlined above, an 
account will be given of a project of The George Washington University 
Human Resources Research Office (HumRRO) carried out at one of its field 
units, the United States Army Armor Human Research Unit at Fort Knox, 
Kentucky. The purpose of the project, known as Task SHOCKACTION, 
Was to build a revised course of instruction for the armor crewman. The 
Project was undertaken during a period of years from 1955 to 1959. Six 
Subtasks were completed, covering each aspect of the developmental sequence 
outlined above, although the subtasks were not organized to correspond 
exactly with those six steps. While the final subtask of constructing and meas- 
uring the effectiveness of a new training program was foreseen at the outset 
of the work, the various stages were undertaken and reported as individual 
Projects. Certain results of each step, although of primary value in furthering 
the developmental project, were of immediate interest and use to the Army. 

The training program toward which the task was directed was the Army 
Training Program for Tank Companies, prescribed by the Department in an 
appropriate Army document (Department of the Army, 1957). This course, 
taken by graduates of basic combat training (the initial common training 
Program for all new male enlisted personnel), leads to the position of armor 
crewman, a job in which the man is presumed to be capable of acting as a tank 
Commander, gunner, driver, or loader of the main battle tank. The course is 
eight weeks long and is given at several Army installations. It is an important 
Course because it serves. as an introduction to all enlisted training in armor 
nd serves many students, and was therefore judged to be worth a consid- 
€rable investment of research and development funds. A previous survey 
9f training problems in armor (Green er al., 1954) recorded the opinion 
of many armor officers that the training of armor crewmen was not at a satis- 


factory level of proficiency. 


In order to define completely the objectives 
Of the new training program, a number of determinations had to be made 
about the performance of tank crewmen in the operational subsystems repre- 
Sented by various kinds of armored units then on duty in the Army, both in 
the continental United States and overseas. The research team planned a series 
Of studies which would result in job analyses of the four crew positions, 
together with a listing of the knowledges and skills required for each. In addi- 
Ion, a program was conceived in which various assessments would be made 
BE the average existing state of knowledge and proficiency in samples of a 
Ide range of armored units. . 

The first series of studies concentrated on groups of experienced armor 
Personnel in order to determine the actual job duties, knowledges, skills, and 
vels of proficiency in those units which would be expected to be in the 

'Bhest state of combat readiness. It was considered that such units would 


Collection of Field Data. 


318 - Concepts of Training 


display most accurately the nature of the jobs as they existed in the best 
operational systems and as performed by the most proficient personnel. To- 
ward these ends, members of the task team began their preparation by 
enrolling in appropriate courses in the Armor School, by reviewing Army 
manuals and training programs relevant to the tank crewman's duties, and of 
interviewing experienced officers. As a result of these preliminary efforts, 
several tentative data-gathering instruments were prepared, including an Indi- 
vidual Interview Form, Observational Check List, performance tests for cach 
crew position, and an Activities Log on which crewmen were to specify how 
they spent a typical day in their unit. 

For tryout purposes these instruments were used with 100 members of 
an armored cavalry regiment. The instruments were revised on the basis of 
this experience, and a second armored unit was identified, the members of 
which had at least three years of experience in armor and three months’ 
experience on the job. The revised instruments were used with the 25 crews 
(100 crewmen) of this unit, to further test the suitability of the data-gathering 
materials for the main study. 

While these preparations were being made for intensive study of experi- 
enced crewmen, another measure was developed to be used with crewmen 
of a wider range of experience for study of interchangeability of jobs among 
crewmen, since the ability of one crewman to replace another who might 
become a battle casualty is regarded by the Army as an important charac- 
teristic of armor crews. This instrument was a paper-and-pencil test of armor 
knowledge, the 157 items of which were grouped into four parts for the four 
crew positions (MacCaslin, 1956). 

The task team visited Europe during September and October, 1955; 
where 256 experienced crewmen of three medium tank battalions and two 
armored cavalry regiments were studied with the pretested research instru- 
ments. With the tank commanders, for example, the researchers interviewed 
each commander, using the Activities Interview Form as a guide for asking 
questions about various tasks, his ability to perform them, the frequency with 
which he does so, and the degree of difficulty involved. From the 69 com- 
manders studied, a count was made of the proportion who said they had 
engaged in, had responsibility for, or knew how to perform 61 elements of 
their job duties. Because of the large number of job elements, a sample of 
20 elements was selected for proficiency measurement by means of perform- 
ance checks, in which the percentage of men correctly responding on each 
task clement was noted. The daily Activities Log was kept by each tank com- 
mander, and during the week spent with each unit, a ten-item descriptive 
rating scale was completed on each tank commander by his platoon leader. 


Similar procedures appropriate for each crew position were used for 57 gun- 
ners, 60 drivers, and 70 loaders. 


Job Analyses and Knowledge and Skill Specification. From these 
observations, information on knowledges and skills and on proficiency were 
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derived (Baker, 1958). A trial list of job duties for each crew position was 
compiled by listing each job element which was reported by 60 percent or 
more of the experienced men studied. These were submitted to experienced 
staff officers for evaluation and comment, and after conferences with them 
to resolve differences in opinion, a final criterion list of duties was prepared 
for each position. Eight "job duties" were specified for the tank commander, 
four for the gunner, nine for the driver, and five for the loader, of which the 
following are typical: 

The Tank Commander applies basic methods for collecting and reporting 

combat information and applies and supervises troop counter-intelligence 

measures. 

The Tank Gunner fires the main gun and the coaxial machine gun, utiliz- 

ing direct fire procedures. 
Under each job duty were listed "operations." “knowledges,” and “responsi- 
bilities,” ranging in number from 1 to 34. These lists of job requirements 
constituted the “chief product of this first subtask of the program, supple- 
Mented by detailed job descriptions for each crewman which specified the 
behavioral steps in performing the tasks. They provided not only information 
for further training developmental work, but, as a by-product, also made 
available information on the basis of which the Army revised its published 
description of the Occupational Specialty Code pertaining to armor crewmen. 
This represents an incidental contribution from training research to the per- 
sonnel management subsystem of the Army. 


Assessment of Current Proficiency. The results of the field proficiency 
tests were reported in detail (Baker et al., 1958) and formed the basis for a 
Number of conclusions and recommendations, the principal one of which was 
that the beginning course for armor crewmen concentrate on the position of 
Bunner, loader, and driver, leaving the special training for the tank com- 
Mander to a more advanced course. In addition to the use of these proficiency 
Measures in pinpointing particular job elements in need of improved training. 
the results along with those from an administration of the 157-item paper 
and pencil test to a representative sample of active, National Guard and 
reserve units throughout the United States—furnished the Army with current 
information on the state of training and readiness of armored forces. 


of Variations in Training Time. These studies 
ation of the requirement to construct a 
ation on which to formulate precisely 
addition, another study had been ini- 


ns A Preliminary Study f 
nished both the basis for reaffirm 
new training program and the inform 
XC Objectives of the new course. In a 1 t 
tiated, to determine the effect of halving, repeating twice, or repeating three 
times the training in selected armor subjects (Baker, Mathers, and Roach, 
959). This research in the psychotechnology of training was undertaken in 


advance or training-course instruction to obtain data on relevant learning 
aining- s s 


nomena to be used in the terminal phase of engineering design. 
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The increase or decrease in each subject was done by keeping the lesson 
content constant, and varying only the amount of instruction or practice. 
Through the use of comparable groups of armor trainees for each presen- 
tation condition for each of 20 selected subjects, the scores on achievement 
tests in each subject for some 2500 soldiers were obtained, analyzed in terms 
of low, medium, and high aptitude trainees (as determined by Army General 
Classification test scores) and plotted in the form of "learning curves" for 
each subject. Two such figures are displayed in Figure 9.3. The results as a 
whole indicated an increase in the percentage of test items answered correctly 
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Figure 9.3. Achievement in armor subjects by amount of instruction. 
Each point represents the average score made by a group of trainees of 
specified aptitude level. (Adapted from Figures 21 and 22, Baker, Mathers, 


and Roach, 1959, by permission of the Human Resources Research Office, 
The George Washington University.) 


with increased training, but considerably less than expected by Army instruc- 
tors. Even with three times the standard instruction and practice, proficiency 
increased but little, while it decreased in only eight of the 18 skill areas when 
normal training was cut in half. Marked differences of achievement at all 
points along the abscissa were observed for the two extreme aptitude groups 
in most subjects, particularly the critical areas of communications, gunnery, 
driving, maintenance, and tactics. It was concluded that a mere increase in 
repetitive instruction would not produce the desired proficiency, and there- 
fore special instructional techniques would have to be developed, particularly 
for the more difficult subjects. The almost negligible effect of halving instruc- 
tion for some subject matters indicated those areas in which training time 
might be saved for use with new kinds of training on more difficult subjects. 
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Construction of the Final “Training Package." With the information 
gathered over the three previous years, the research team was prepared to 
begin a study of the existing program of instruction (Department of the 
Army, 1957) in terms of curriculum content, and instructional and adminis- 
trative techniques, toward its modification to produce the product desired— 
“a tank crewman qualified to perform the job of gunner, driver and loader” 
(MacCaslin, Woodruff, and Baker, 1959, p. 3). On the basis of information 
gathered during the field studies, which showed that tank commanders do not 
achieve their position until after considerable on-the-job experience, the Army 
Concurred in a recommendation that the objective of training tank com- 
manders be omitted from the new course under development. However, as a 
by-product of this task, members of the task staff prepared the Tank Com- 
mander's Guide (Cook and Baker, 1959) which brought together revelant 
information from 90-odd Army publications and which was made available, 
through a civilian publishing company. for general Army use and as a text in 
à subsequent study on tank-commander training. 

In planning the details of instructional procedures for the new course, 
Seven psychotechnological principles served as guidelines: 

l. Adherence to training objectives. This was accomplished by pre- 
Paring lessons in advance for presentation by instructors, production of train- 
'ng films for some teaching points, recording certain presentations of complex 
€ssons, and the usc of picture guides, described below. 

2. Emphasis on relationship of tasks to each other and to the program. 
The new course was designed to begin with a specially prepared orientation 
film on armor and on the training program. Each section of the program was 
troduced and summarized in an integrative manner. 

3. Emphasis on learning by doing. Since the job descriptions broke 
down cach task into steps, it was possible to direct cach trainee through these 
Steps under supervision and to correct his mistakes, a procedure facilitated 

Y instruction in small groups. . . 
$ 4. Promoting understanding. All lesson materials were prepared in word- 
P8 and illustration with the aim of maximum communication to the trainee. 
P particular, the job descriptions were used to develop a picture guidebook 
Or each of the three crew positions. These books provided pictures and 
“'mple verbal statements of many job procedures and were put together after 
pirical testing of much of the material on trainees available at the Center. 
ese Were used as texts throughout the course (U.S. Army Armor Human 
“Search Unit, 1957; 1958a; 1958b). 
| 5. Knowledge of results. Performance achievement tests on each block 
T struction were provided, and trainees were told their scores. When firing 
tem TÈ range, information on where his round went was transmitted to the 
“mee as quickly as possible. } . a 
9. Motivation as an integral part of traning. On the first day of train- 


i : 
ng, Students were shown a specially prepared film, /ntroduction to Armor, 


322 - Concepts of Training 


were given a chance to examine tanks and witnessed a firing demonstration, 
all designed to answer questions such as, "Why am I here?” and “What will 
I be doing?" The attempt was made to present the role of armor in modern 
warfare. Scheduling of training for individual participation, for remedial 
Saturday afternoon make-up sessions for poor performance, and for organized 
athletics and recreation was accomplished to enhance conditions of positive 
motivation. Additional general orientation to armor and the course was given 
during the first day of the third week. 

7. Realism in training. Insofar as possible, training situations were de- 
signed to reflect conditions of combat. Activities not related to combat had 
been removed from the curriculum. The kinds of targets used in firing, the 
position and sequence of firing, and the courses over which student drivers 
operated their tanks were chosen to reflect combat situatons. A night firing 
problem was added. 

To implement these general principles, administrative changes in the 
management of training were accomplished. The training company was di- 
vided into four platoons who lived, went to school, and performed other 
duties together, which resulted in only a few absences from class. The instruc- 
tors for the experimental program were relieved of other duties and were 
trained for a two-week period prior to the experimental run. Those men 
normally held over from previous training classes to act as tank commanders 
for a new group of trainees were designated, for the new course, as “Assistant 
Instructors,” rather than *Held-Over Tank Commanders,” were promoted 
earlier than usual and had their duties arranged to permit them to stay with 
their crews throughout all training. To make this possible, it was necessary 
that ten additional men be retained for the duration of this study to remain 
with the tanks during periods of shop maintenance. The normal number of 
tanks were used but were driven by students more hours than in the standard 
course; because of scheduled economies in unnecessary driving by adminis- 
trative personnel, less gasoline was used than in the standard course. The 
experimental company used fewer rounds of expensive ammunition for the 
main tank gun than in standard training, with some small addition of the 
less expensive machine-gun ammunition. 

The over-all experimental program turned out to require 274 hours of 
regular scheduled training with four hours of additional concurrent training, 
while the standard program called for 352 hours of regular and 28 hours of 
concurrent instruction. This reduction of time from eight to six weeks was 
not a principal objective at the outset of course planning, but appeared to be 
possible during the detailed allocation of training time and facilities. 

It is apparent that a great many variables believed to affect training were 
altered in some way for this experimental purpose. The results, therefore. 
represent the combined effect of particular curriculum and administrative and 
instructional conditions, all engineered into a single package. In the develop- 
ment of psychotechnology, it is desirable to determine the effects of each, 
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varied while others are held constant, as well as their interactions, but such 
research requires time and resources not easily provided. Therefore, the eval- 
uation is of the single combination of factors, weighted in terms of the best 
judgment of the research staff and combined in ways designed to be replicable 
by the Army in possible repeated uses of the new program. 


Construction of the Final Proficiency Test. The Armor Mastery Test 
was built by a separate research group not associated with other members of 
the task team who developed the training program. The test battery com- 
Prised 21 subtests, made up of 405 items, constructed as mastery items— 
that is, they were designed to ascertain only whether the trainee had learned 
the essential skill well enough to perform a given job satisfactorily, and not 
to measure relative degrees of skill. Nine subtests were devoted to gunnery, 
Six to driving, and four to loading, and two were general tests on the pistol 
and submachine gun, weapons to be used by all crewmen. Each subtest 
Was of the performance type. In developing the tests, only key skills for each 
Crew position were used, and practical considerations of time, equipment, 
and feasibility were guidelines. During the preparation of the tests, Army 
instructors criticized them, and they were tried out on trainees undergoing 
Tegular instruction. They were revised and standardized for administration 
by noncommissioned officers not associated with the research. In the experi- 
Ment, conventional firing schedules were used because the criteria for new 
firing procedures, later developed by HumRRO, had not yet been fully devel- 
Oped (Woodruff, Stark, and Willard, 1962). A new driving course was pre- 


Pared for the test. 


Evaluation. The final measurement was performed with two training 
Companies of about 160 men each, made up by the assignments to each in 
terms of scores on the Army General, Mechanical Aptitude, and Verbal 

ests and in terms of educational level, race, and geographical origin. The 
Control company took the current standard eight-week training program and 
© experimental company, the new six-week course. The former was started 
two weeks ahead of the latter, so that both companies were given the Armor 
astery Test during the same final week, alternating mornings and afternoons 
of each day for each company. Other precautions taken to eliminate bias on 
* test procedure included: selection of comparable officer or noncommis- 
Oned-officer cadres for each company; instructors for the experimental com- 
any chosen from those rated by supervisors in the Armor School as average 
Clow average; assistant instructors (and hold-over tank commanders for 
i^ Control group) chosen from those rated in the upper third of their train- 
g Companies; and a briefing given to each company on the final day, 
Plaining that they were part of an experiment, but without mention of 
Mpetition with another company. 
he results, in terms of the Armor 


Si 
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Mastery Test as a whole, showed a 
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slight superiority of the experimental group in terms of total score (experi- 
mental, 295; control, 285). Because the subtests differed in length and diffi- 
culty, a more meaningful analysis was made in terms of these. The experi- 
mental company was superior in 11 of the 21. The control company excelled 
in three. In seven the results were the same. In general, the 11 tests on which 
the experimental company showed superiority involved the more complex 
skills involved in gunnery, driving, and communication. Differences reflected, 
in large part, certain differences in emphases between the two courses. In 
addition to these results, the scores on the paper-and-pencil Armor Proficiency 
Test, used earlier in the field studies, showed no differences between the two 
companies. 

The major conclusion was that the experimental six-week program pro- 
duced trainees essentially equivalent in proficiency to graduates of the stand- 
ard eight-week program. In considering this new training program for adop- 
tion, the Army considered the 25 percent gain in training time in comparison 
with the costs. This is, of course, a decision to be made by the proper military 
authority (as equivalent to the manager of the parent system). But it was 
important that the research report tabulate and discuss the differences in 
management costs in terms of the several enumerated factors. At this time of 
writing, it can be reported that the experimentally produced guides for the 
gunner, driver, and loader have been published as Army Training Circulars, 
new firing tables resulting from this study have been adopted, and a new 
training program, lengthened to the original eight weeks to comply with 
existing federal statutes and to provide for additional training, has been 
adopted by the Army on the basis of this developmental program. 


Ej 


THE MANAGEMENT OF THE LEARNING PROCESS 


This section is concerned with those decisions that are called for in the 
steps in the training development sequence designated above (and in Figure 
9.2) as “Determination of Training Objectives” and “Construction of the 
Training Program.” This account will be confined to the kinds of choices that 
have to be made; a few suggestions about the technological information avail- 
able in support of these decisions will be added. To go further would require 
the study of a number of particular developmental efforts of diverse and 
unique interest. Generality in this field inheres more nearly in the specifica- 
tion of the orderly array of decisions that must be made than in rules as to 
how to make them in specific instances. At the outset, it should be emphasized 
that decisions in these three areas must be made simultane 
are mutually interactive and supporting. Certain kinds of 
a preference for certain programming techniques 
enhanced or reduced in terms of various choices 


ously, since they 
curricula indicate 
. and motivation may be 
of curriculum material an 
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programming techniques. Since the learner is an active system, his manage- 
ment during the training process is to be done with a series of strategic and 
tactical decisions, simultaneously comprehending what he is to learn (cur- 
riculum ), why he should learn it (motivation), and how he should learn it 
(programming). 

Since the training subsystem is designed for the primary purpose of 
Supplying the operational subsystem with capable human components, it fol- 
lows that the chief concern in designing training, in psychological terms, is 
With transfer of training from the training to the operational situation. Much 
Of what has been said before about feedback of information to the training 
Subsystem from the operational subsystem, realism in training, and the pat- 
tern of gradual differentiation over the years of the typical training subsystem 
from the parent system, all point to the essential requirement that the learning 
Must transfer to the place where it will be used. As Gagné and Bolles (1959) 
have Pointed out, a good deal more is known about the conditions for efficient 
acquisition of learning than for transfer. 


Major Decisions to be Made in Choice of Curriculum 


The term curriculum is used here to denote the subject content of a 
Course of instruction designed to qualify a man to perform a job in the opera- 
tional subsystem. The instruction may be given in a school, on the job, or 
In à combination of the two. Choices about this content involve major deci- 
Sions of where the training may best be done, and they involve determina- 
lions, if not already made, of the job aids which will be available in the 
9Perational subsystem. 

The Derivation of Training Objectives. The clarity and adequacy of 
the specification of course objectives is primarily dependent upon the com- 
Pleteness and accuracy of the determination of knowledge and skills. If this 
'S Well done the maximum curriculum coverage is readily apparent. Choice 
of curriculum material is then governed largely by particular considerations 
9" time, cost, and equipment availability. For example, if the knowledge and 
Skill Specification shows a number of closely related items, a single unit of 
"struction may suffice. Again, all possible inputs of stimulus situations 
Which occur on the job can probably never be economically reproduced in 

training situation, and therefore representative inputs will have to be 
“lected. "These may be chosen if the knowledge and skill specification is 

ased on the frequency of occurrence of various input conditions on the job. 
ies a frequency count, for example, is quite possible in the study of a 
Pairman’s job, where the number of different malfunctions encountered may 
es tallied, The objectives of the curriculum are thus set in terms of the 
“quisition of the most important, the most frcquent, and the most representa- 
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tive of the knowledges and skills, and perhaps those most likely to be 
required when the man first goes on the job. 

Certain general rules can be stated for the specification of course objec- 
tives which may encompass a number of specific choices. The training course 
objectives should be: 

1. derived from the job, in terms of what is actually required for job 

performance; 

2. arranged in hierarchical order, for the program as a whole and for 

each unit of instruction within the program; 

3. cognizant of the knowledges and skills already within the repertory 

of students before the training begins; 

4. reasonably within the achievement level of the students; 

5. communicated to the students. 

In following the third rule, for certain programs it may be necessary tO 
prepare refresher training for some students whose prior repertories are 
inadequate, as, for example, training in mathematical operations before 
admission to technical courses requiring computation. The fifth rule is based 
on the fundamental characteristic of the human component in a system—his 
ability to modify his own behavior. Unless the objective toward which the 
modification is to be made is clear to him, one of the most important resources 
available to the training manager is overlooked. 

Specific rules for the translation of knowledge and skill specifications 
into technical training course content have been described in Chapter 6. and 
also by Hoehn (19602), who has illustrated them in the development of an 
electronics maintenance course. As previously stated, perhaps the most im- 
portant single contribution to the development of training through research 


has been the determination of methods for the formulation of objectives of 
instruction. 


Manuals versus Memory. In planning the curriculum, a very impo- 
tant distinction must be made between what the man must be able to do 
from memory and what help he can expect from job aids, manuals, check 
lists, and other devices to provide cues for his action. The precise nature of 
such job supports needs to be foreseen at the time subject matter for school 
training is being selected, so that school training can be built around the sup“ 
ports which will be available on the job. It is often true, for example, that an elec- 
tronics technician has the information he needs for troubleshooting within 
his manuals and schematics but has not learned how to use them in an 
efficient manner. If the design of these manuals can be anticipated at the 
same time training is being developed, training time will be used effectively» 
and the employment of job aids will be precise and rapid. In connection with 
the preparation of guidance for choosing training course content, Hoehn has 
suggested procedures for the parallel development of handbooks to be used 
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on the job, especially for the presentation of hard-to-memorize performance 
Sequences, infrequently used procedures, and specific equipment character- 
istics (Hoehn, 1960b). The design of job aids was discussed in Chapter 8. 


School versus On-the-Job Training. Another important decision which 
must be made early in the training development cycle is between the acquisi- 
tion of knowledges and skills in the school or on the job. As has been previ- 
ously indicated, training in the operational context has the advantage of 
realism, thus minimizing the transfer problem, and also allows for some 
gainful employment of the new worker in the operational subsystem. In some 
Cases, the decision may be between the school and the job situation; in others, 
à judicious integrated combination may be made of the two. Some considera- 
tion of the advantages and disadvantages of on-the-job training was given in 
the earlier discussion of the apprentice system. 

One of the most common difficulties with on-the-job training is that little 
Systematic attention is paid to it. New employees may be put into the job 
Situation and given sporadic explanations, instructions, or help by older em- 
Ployees, but without a planned program for learning. Usually, the older 
€mployee does not have the time to devote to instruction or special skills in 
Planning learning experience. To meet the need for such programmed on-the- 
Job training, Woolman (1960) developed a program for training of guided- 
Missile Operators on site. The instruction was prepared for men who report 
to Nike sites for duty directly from basic combat training, without any rele- 
Vànt experience. The development of this program generally followed the 
Procedure outlined in the previous section. The Operational Context Train- 
ing, as it was called, “. . . was to provide on-site training, flexible enough to 
Produce operator skills without interfering with battery operations. Instruc- 
tion was conducted in “training units” made up of an experienced operator 
and two trainees. It was scheduled around the regular duties of the crewmen, 
and was flexible enough to utilize parts of the equipment which were free 
rom Operational use. K “training guide” was prepared which not only pre- 


Sented in nontechnical language the standard operating procedures for all 


Positions, emphasizing safety factors, but also provided for records of student 


Progress on each block of instruction. For each operating procedure, the 
Instructor. was furnished a six-step training method which began with his own 
Presentation of the over-all picture of the procedure and followed through 

*ps of instructor demonstration, instructor talk-through, trainee-instructor 
‘alk-through and finally, trainee practice. A set of proficiency standards was 
Provide for each procedure. In a carefully controlled study, the field train- 
he În this operational context proved superior to previous ia peer train- 
wae and produced, at far less cost, men with proficiency equal to graduates of 


9a training. ; ; E 
It is probable that great advances, both in the efficiency of training and 
= e 
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the performance of certain kinds of jobs, will be made through a judicious 
division between school and on-the-job training. The latter seems very effec- 
tive when it is associated with a careful division of knowledge and skill 
support between training, on the one hand, and precisely designed manuals 
and other job aids, on the other. Following a comprehensive task-and-skill 
analysis, such a delineation of a program for preparing and supporting the 
human component in the system will yield important results. 


Motivation of Trainees 


The point of view has been adopted that the training subsystem is part 
of the parent system and that interest in learning is but a special case of an 
interest in performance. Therefore, it follows that the first examination of 
the conditions for motivation of training would be in terms of the gencral 
conditions for motivation of work in the parent system. In other words, the 
trainee is an employee of the organization or member of the parent system 
whose current work assignment is to master a training course. Consideration 
of trainee motivation will here be divided into two parts: factors which affect 


general employee motivation, and particular motivational conditions which 
promote learning. 


Employee Attitudes and Motivation. The new employee entering 2 
company training program or the recruit entering basic training is making an 
important change in his life. The enthusiasm with which he begins may range 
from a keen anticipation, following a deliberate choice among several attrac- 
tive employment opportunities, to a reluctant response to the notification from 
his draft board. His initial attitudes toward his first training program are 
therefore already determined. The early days of the training program may 
give promise of the realization of his highest expectations or of his worst mis- 
givings. As the training progresses, he forms attitudes, not only about the 
training program and the knowledges and skills he is expected to acquire, 
but also about the parent organization which he has joined, as has been 
quantitatively demonstrated in a study of motivation for Army technical 
training (Heilman, Osburn, and Hausknecht, 1955). The way training is 
conducted, the apparent attitudes of instructors, and the facilities provided will 
offer cues by which the trainee formulates attitudes toward the parent system- 

Since the trainee is already “at work,” it is reasonable to assume that 
many of the conditions for good motivation which have been found in indus- 
try (Viteles, 1953; Maier, 1955, Chs. 4, 5, 13, 14; Industrial Relations 
News, 1960) and in the military services (Stouffer, 1949-1950) are generally 
applicable to the training situation. Perhaps one of the most important findings 
of recent research on motivation for work is the importance of factors other 
than financial reward. Many stndies point to the fact that a feeling of achieve- 
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ment, interesting work, and recognition of accomplishment are more highly 
desired than pay, provided the latter exceeds a certain minimum level. The 
recent study by Herzberg and his associates (1959) on employees in engi- 
neering and accounting work found that motivational factors could be classi- 
fied as "satisfiers" and "dissatisfiers." Among the former were such factors as 
achievement, recognition, the work itself, responsibility, and advancement, 
Which could take positive values well above the zero, or indifference, point. 
The latter included such things as supervision, salary, and working condi- 
tions, which could be perceived as varying negatively below the zero point. 
The chief implication of the research was that, even if dissatisfiers were 
Teduced to a minimum, satisfiers would have to be present in a finite amount 
to keep the employees from seeking other positions. Such results point to the 
importance of the work itself and its place in the parent system as a deter- 
miner of motivation. f 

It would seem important to capitalize during training on the same kinds 
Of positive factors which will be important for good work throughout the 
Parent system. Insofar as it can be assumed that the individual wants to get 
ahead in the parent system, doing work in which he has some interest, train- 
ing can be conducted in a manner to allow some realization of this desire. 
Further, it seems reasonable that those incentives and motivational techniques 
Which can be used in a particular operational subsystem to promote good work 
Should also be used in training, because they would not represent artificial 
Supports, to be removed when the man goes on the job. Attitudes toward the 
"work" of learning would transfer to the “work” of production. Looking at 
It another way, it is clear that an employee is expected to continue his learning 
9n the job. with or without formal on-the-job training, so that, from the 
Point of view of attitudes and motivation, learning in training is much the 
Same as working at the job. Positive attitudes, reflected in a desire to learn, 
Should be cultivated and reinforced during training in the hope that they 


Will be taken into the job situation. 


Motivation for Training. Within this perspective, both the motivating 
Conditions for the training situation as a whole and those particular factors 
ich affect the detailed acquisition of knowledges and skills are of interest. 

;. En the learner himself is regarded as a system, two characteristics that are 
portant for his motivation as a learner become apparent. The first is his 
lity to acquire goals or purposes by which he governs the application of 
fort, The scennd is iw fact that new outputs from this system come about 
Tough a sequence of inputs and outputs, a situation summarized by the 
cepted principle that the learner must be active. Further, the selection and 
“quisition of these new responses or modes of behavior isa function of the 
Poses or goals internalized by the learner. While it is recognized that peo- 


Ple are often misled as to the nature of their cwn motives and the goal objects 
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In the military establishment and many other training situations, initial 
attitudes of trainees toward the parent system and interest in the knowledges 
and skills to be initially acquired do not offer the training manager much on 
Which he can build a positive program of incentives. For specific learning 
Situations in the classroom and laboratory. the evidence seems clear that 
reward is more effective than punishment, particularly since punishment alone 
does not offer the learner any cue to the correct response. In military train- 
ing, and probably also in training for various positions in large organizations, 
there is a type of conformity required in many kinds of behavior, more or 
less related to the immediate job, which is brought about through the exer- 
cise of various kinds of social controls of a positive or negative sort. Punish- 
Ment, as meted out by authorities OT as developed by group attitudes and 
Sanctions, is clearly operative and influences behavior. Information is not 
Teadily available on the extent to which the learning of specific knowledges 
and skills is so programmed in the military and industrial training that a form 
of punishment regularly follows an incorrect response. Nor, for that matter, 
's the writer aware of any systematic survey of the use of rewards. It is true 
that rewards and punishments are meted out in the practical training situa- 
tions, often without precise relation to specific correct or incorrect acts. As 
Noted in The American Soldier (Stouffer, 1949, Volume I, pp. 410-449), 
incorrect, nonconforming acts are easier to recognize, 1n the absence of 
reliable performance criteria, than are outstandingly good performances. This 
standable impatience of training personnel bur- 


fact, together with the under: 
dened with heavy schedules and reluctant students, leads to a frequent use 
hat, in a sample drawn at the 


Of some form of punishment. Stouffer reports t \ 
end of World War II, 46 percent of officers and 67 percent of enlisted men 
agreed with the statement: “The main reason most soldiers obey rules and 


regulations is because they are afraid of being punished.” The real effective- 
Ness of punishment—or better, the conditions under which it is an effective 
and wise technique for the management of learning—is yet to be determined. 
A discussion of its role in promoting discipline is beyond the scope of this 


chapter, 
One other point in connection with motivation for learning should be 


noted. This concerns the problem posed for training by the possible presence 
of real danger during employment in the parent system. High-voltage elec- 
tical sau pet high-speed moving parts, and hazardous missions of one sort 
and another are characteristic of many jobs, both military and industrial. 
Although reliance can often be placed on volunteers for many such occu- 
Pations, there are still problems involved in the control of fear. At present, 
we do not know, for example, at what point in the learning sequence it is 
best to introduce conditions promoting fear and stress, in order to insure that 
the behavior will be properly elicited under such conditions in the real-life job 
Situation. Dependence on overlearning is perhaps the most common current 
Solution to this problem. 
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The Programming of Instruction 


The adaptation from the language of computer operation of the term 
“programming” offers the connotation of providing for an orderly, sequential 
set of operations by which the learner is to achieve a new behavioral output. 
The use of the term affirms the assumption that the learning process is an 
orderly one and expresses the conviction that it will eventually be understood. 
As yet, however, learning theory has not made explicit just how the process 
works on either the human or the infrahuman level. 

For the reader interested in learning theory, 
the level of discourse which is appropriate for tr. 
ment of the learning of certain specified samples 
in groups, is the practical problem. The principa 
those arrangements of antecedent conditions 
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process; the points of view of individual writers gi 
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facilities may require a fairly rigid prescription of the training program, so 
that it may often be best to present as much of the instruction as possible by 
previously prepared and fixed media, insuring at least a minimum compre- 
hensive and orderly coverage of subject matter. 

The vehicles for programming are the course outline and the lesson plan. 
The kinds of decisions which have to be made in the development of these 
Were summarized from experimental studies of military training during World 
War II by Covner (1946). His list of choice points in building a course 
outline serves as well today as it did 15 years ago. The outline builder must 
answer questions about the time required for learning each knowledge and 
skill by the average trainee; the relative difficulty of topics; the optimum order 
of subject matter; the presentation of theory in relation to practical informa- 
tion; the allocation of time for practice, drill, testing and review; and the 
extent to which the outline may be altered to meet the characteristics of the 
individual learner. In preparing the detailed lesson plan, Covner indicates that 
the trainer must make explicit the purpose of each session, the necessary 
Preparation of the instructor, the plan of presentation, the provision for stu- 
dent practice, and the method of conducting the review and summary in which 
the lesson is integrated with those coming before and after it. For the presen- 
lation, he indicates that the introduction of technical terminology, use of 
training aids, and the questioning of students should be specifically anticipated. 

The making of these and similar decisions constitutes the work of pro- 
Sram planning. What psychological principles are available for guiding the 
developer in making his choices? Summaries of such principles may be found 
in textbooks on learning (McGeoch and Irion, 1952; Kingsley and Garry, 
1957) and in articles on training (Wolfle, in Stevens, 1951; Gagné and 
Bolles, 1959) as well as in chapters on learning in textbooks of general and 
experimental psychology. These principles constitute a kind of policy guidance 
for the programmer, rather than specific rules; like other types of policy 
Buidance, they do not all apply to a single situation, and in some cases they 
May call for incompatible choices in a particular application. The reader is 
familiar with many of these, of which the following are typical: 


1. Meaningfulness of material promotes learning. 
ould be kept active in making responses. 
o be more eflicient than massed practice. 


sults should be provided. 


2. The learner sh 
3. Distributed practice tends t 
4. Immediate knowledge of re 
5. Stimulus material should be varied. 
6. Accurate records of the learner's progress should be maintained. 
While | Early guidance is useful. 
lle principles of this sort have 
Verbal learning, they probably appl 
Chaviors which have some symbolic component. 
Broad and general as are the above kirds of principles, they still offer 
Useful guidance. Many existing training programs could be improved by their 
Careful application, even though they are not very precise. A great many 


m 


been stated on the basis of the phenomena 
y to the acquisition of a variety of 
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specific studies have been made on the acquisition of various knowledges and 
skills, as well as on the learning of different kinds of tasks. To examine this 
literature would require too much detail for the purposes of this chapter. It 
may be observed, however, that existing information is difficult to organize 
because categories for classification of both tasks and the relevant variables 
are still uncertain and because many of the existing training studies, done from 
the practical engineering point of view, necessarily confound variables whose 
interactions are unknown. As a first stage in the evolution of this technology. 
it seems likely that a number of provisional generalizations and specific guid- 
ance statements will be offered, to be revised and elaborated with further work, 
as has been begun by Hoehn (1960c) for maintenance training. Also, Jones 
(1961) has examined the specifications of several hundred Army enlisted jobs, 
classified the requisite knowledges and skills into several empirically useful 


categories, and suggested some important underlying dimensions for generality 
in training research. 


Another series of decisions which need to 
ing program concerns the media through 
- This section will be confined to an identi- 


he also plays the role of s à 1 E 
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dium—the trainee's fellow students. Surely the attitude of the student group 
toward the training program, its purpose, and its conduct is of great influence 
on the effort and enthusiasm with which the individual learner attacks his 
Work. The group atmosphere as well as the predispositions of new students 
may well be a function of the instructor's attitude and behavior. The frequency 
and intensity of student discussions, as well as study and work outside the 
Classroom, may depend in large part on the climate of opinion among stu- 
dents as a group; they can learn much from one another. As Melton points 
out, little research has been done on how to promote an atmosphere con- 
ducive to learning among the students as a group. 

The inanimate media are many, and their characteristics are discussed in 
Chapter 10. Some are offered by publishers and manufacturers, with little or 
no evidence of their usefulness for promoting learning. Claims often rest on 
nothing more than “face validity.” The criterion for the evaluation of a train- 
ing device as a medium for improving performance is, as Gagné has pointed 
Out (1954), the degree of transfer of training to the operational situation. 
Such a criterion must be applied to all media, in systematic comparative 
Studies for the improvement of technology, and when used in a training 
Program, for the evaluation of a particular, developed training package. 


Aptitude of Trainees. While most of our discussion thus far has 
assumed that the task of training management is to provide instruction for a 
Broup of students of somewhat heterogeneous background and aptitude, it is 
Possible to divide trainees into relatively homogeneous groups when there are 
large numbers of students and valid classification tests are available. When 
this is not possible, it is important to choose methods and media which will be 
effective with a wide range of aptitude, particularly with men with intelligence- 
test or other test scores lower than may be ideal for certain kinds of training. 
As indicated in Chapter 7, there is a great deal of information on the relation 

tween aptitude variables and achievement in different training programs, so 
that selection and placement in terms of background variables is done with 
Practical economy. A good deal less is known about the relative appropriate- 
Ness of various methods of programming training for different aptitude patterns 
and levels, A promising basic research study has been made by Allison (1960) 
9n the relation between scores on a number of tests of human ability and rate 
of learning on 13 different short learning sequences. He finds significant rela- 
tions between factors within the learning variables and factors representing 
the fundamental human abilities. This kind of research may lead to differential 
Programming for learning in terms of various aptitude patterns. 

A few studies have attempted to relate different teaching methods to 
aPtitude differences. Edgerton (1958) showed that certain predictor variables 
Correlated differentially with achievement in naval technical training taught 
Under a “rote” method and under a “why” method. The results were clearer 
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i ircraft familiarization than with material involving mechanics, mathe- 
ben aiiora ics. I revious investigation, the same author (1956) 
DM n i: hing *how to do" before the 
found some advantages for a method of teaching "ho Á i E ded 
“why” parts of a naval aerographer's course, for men with higher m mer 
ability than rote memory, while the reverse tended to be true for t pod 
higher aptitude scores in word fluency and reasoning. A somewhat x: € 
approach to the programming of training in terms of different abilities e in 
taken by Fleishman and his associates (Parker and Fleishman, 19 de 
designing a motor-skill training program. From an analysis of the relati ^ 
between different abilities to achievement scores at successive stages e 
practice, various types of instruction and verbal guidance were given a 
different times throughout the course of learning a pursuit rotor task. Students 
$0 trained showed an increase in performance of 39 per cent over those 
trained by the “common sense" method. 

A rather extensive study was undertaken to evaluate several methods for 
programming learning to benefit men whose aptitudes fell slightly below "T 
normal requirements for admission to Army technical training courses (Gol- 
fard et al., 1960). In the early part of the study, simple techniques—such as 
increasing the number of repetitions of important points in the lesson, improv" 
ing training aids, or giving students mimeographed course notes—were applied 
to short segments of the course. These had small favorable effects xn 
showed up in immediate mastery tests, but tended to disappear by the end 0 
the course. Also, these chan e no more effective with the 


lower- udents. A more comprehensive 
kind of Teprogramming, involving the first three weeks of an electronic" 
repairman course which were devoted to basic electronics, was developed in 
iple, elaborated and tried out earlier bY 
Shoemaker (Shoemaker, Brown, and Whittemore, 1958; Brown et al., 1959): 
rather sharply with conventional 
training, in that the emphasis is on a “whole to part” rather than a “part t° 
onal training begins with laws an 
Circuits to a final combination i? 
operational equipment. The functional context method, on the other hand, 
dent with specific equipment. Prin- 
hey become relevant in the study ° 
a succession of electronic components, so that the student learns the operatio? 
of the principles in the context in which they Operate. In this study, the per 
formance of students trained by the functional-context method was contraste 
with that of a comparable group of men Biven the standard instruction bY 
means of a basic electronic test battery. 
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Classification Battery, the functional-context technique had a different effect. 
For those groups at the lower and intermediate levels, the experimental group 
made about 10 per cent higher scores, while for the upper groups (above an 
aptitude score of 130) the standard instruction resulted in some 6 per cent 
higher scores. Apparently the lower-aptitude groups had trouble with con- 
ceptualizing general rules and facts before they were embedded in a relevant 
operational context, while the upper-aptitude men scemed to do slightly better 
when the principles and formulas came first. 

This last finding suggests that there may be certain techniques which are 
particularly effective for trainees of high aptitude. For example, in a study of 
Army basic combat training (Cline, Beals, and Seidman, 1956) it was shown 
that high-aptitude men learned as much military information in a four-week 
accelerated program as comparable men did in the standard eight-week pro- 
gram. Special techniques of programming were employed, including an evening 
study period by squad members, using mimeographed material which reviewed 
the day's work and previewed the next. These may have facilitated the learning 
by high-aptitude men, although the experiment was not designed to isolate 
this factor. While there is considerable promise of gains in training time by 
adaptation of programs in terms of aptitude, often practical difficulties arise 
in segregating trainees into homogeneous groups, so that these potential 
benefits have not often been realized, at least in military training. The “multi- 
tracked system” of public school education seems to produce improved learn- 
ing, associating, as it does, appropriate types of programming with each track. 


Automated Instruction. The most dramatic and promising effects of care- 
ful Programming are appearing in the field of automated learning. The current 
status of research on this topic has been conveniently brought together in the 
volume of readings by Lumsdaine and Glaser (1960), which includes most of 
the important original papers by Pressey and by Skinner, together with a 
Tepresentative sample of the current work. Because of the WR Eee and 
timely coverage offered by this volume, no attempt will be made to discuss 
the subject in detail in this chapter. Rather, we will add only a few comments 
about the promise, as well as the problems, the technique holds for the broad 


field of training. 
If the total program of i 


automated, j he mos 
, it would represent the : id pr 
Complete cycle of training development as outlined earlier in this chapter. 


The total kit of program materials, together with the device by which they are 
9 be presented, would constitute the final training package resulting from all 
the Tesearch ensiiesng, trials, and tests of the developmental cycle. Such a 
Program pishi be expected to achieve many of the objectives of the training 
velopment stated earlier. It would present a uniform coverage of subject 
Matter Whenever and wherever employed, would be independent of instructor 
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few programming experts to master à variety of subject matter. Finally, the 
instructor, with automated facilities at his disposal, may become a full-fledged 
training manager. 
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PLANNING A TRAINING PROGRAM TO INSURE 
the presence of suitable skills and knowledges in system 
personnel is an effort which requires a development 
program of its own, but which must be closely inte- 
grated with the development of the system itself. Based 
upon the concepts described in the previous chapter, the 
procedures of training generate the need for specialized 
types of equipment generically called training devices, The present chapter 
provides an account of how requirements for training programs and devices 
arise within the context of systems, and how the resulting designs are made 
compatible with system functioning, on the one hand, and with the purposes 
of training, on the other. 

The goals of individual trainin 
training") can both be seen to pr 
varieties of these devices and the 
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TRAINING PROGRAMS 
AND DEVICES 


William C. Biel 


The application of new developments in technology to meet the needs of 
commercial and of military and other governmental organizations for better 
Performance has resulted in a variety of increasingly more complex man- 
Machine systems. These systems create a major problem of training for the 
many persons who are to function within them. The many technical innova- 
tions, and the high costs associated with system development, production, 
installation, and operation have contributed to an increased emphasis on rapid 
and effective means of training large numbers of people. To realize the maxi- 
mal value of these systems, human performance must be brought to a high 
level of proficiency and maintained at this level. Not only must individual 
Proficiency be established and maintained, but in most systems people must 
be trained to work together as a team before the system can become opera- 
tional. In some of them, of which military systems provide examples, much of 
the training must be directed toward the handling of emergency situations 
Which do not often occur under normal conditions, but which are critical for 
effective operation when they do occur. 

In situations where a more complex man-machine system is replacing one 
already in existence, the new one does not always have characteristics such 
that operational, maintenance, and support personnel can make an easy 
transition from the old to the new. In fact, new systems are frequently quite 
different from old ones, and many of the jobs require new and different skills. 
This situation creates the need for highly specialized transitional training for 
those who have had experience in older related systems. For example, needs 
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i have been created by the change from piston-engined to jet aircraft, 
d y the changes from aircraft to missiles, rockets, and space vehicles. 
Ta bai A systems, which require that much information be ors 
processed before decisions can be made, there is growing reliance on d 
speed digital computers (compare Chapter 3). A computer —- epi 
to process and use more information faster than would otherwise e the €: 
This information is summarized by the computer and displayed at on et 
points to the monitors and decision makers. Learning to make the rapid, 
accurate command or management decisions required by a complex computer- 
ized system is a new experience for most persons. uu 

Training programs and their associated training devices are essential in 
preparing personnel for system tasks. These programs and devices provide the 
advantage in meeting the problem of training large numbers of persons to 
operate a system. To plan for training, to make critical and timely aecison 
affecting training, to design training programs with associated training aids and 
devices, should all be integral actions in total System development. The proper 
use of such programs and devices is a matter of major concern for the effective 
employment of a system. 

The aim of this chapter is to describe the factors th 
the design and development of training 
vices for a system. To begin with, we J 
categories of training, individual training and team training, and shall describe 
briefly the kinds of training devices which may be designed for each of these 
purposes. We shall then consider the nature of planning for the development 
of training programs and devices, Finally, a section will be devoted to some of 
the psychological Considerations which are relevant to the design of such 
programs and devices. 


at need attention 1n 
programs and associated training de- 
shall provide an account of two major 


E TYPES OF TRAINING AND TRAINING DEVICES 


system training. 
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Individual Training 


Individual training is directed toward the establishment of the capabilities 
of individuals so that they can perform on the job to an acceptable level or 
can progress to the next phase of their training. This process, when applied to 
individuals on a job which interrelates with other jobs in a system, is frequently 
described as training the individual as a component in the system. There are 
many system jobs which have as their major requirement that an individual 
must learn to operate and use equipment. In such instances the man and his 
equipment together constitute a component in the system, because there is an 
input (or inputs) of information to this component which must be processed, 
and there is a specific output from it. (Davis and Behan provide a further 
discussion of this topic in Chapter 13.) For this reason the training of an 
individual to operate his equipment can be called component training. 

In the design of such training, it is necessary to identify those skills, con- 
cepts, and information which are needed by the trainee and to estimate the 
level of performance to which they must be learned for the trainee to perform 
his task in the system. Where appropriate, the training plan should include an 
assessment of the initial knowledge, aptitude, and skill level of the student. 
The purpose of the training program or device, then, is to change the level of 
the student from some initial status to the level required by the system task. 
Further, in planning the techniques to be used in establishing these capabilities, 
the planner must also estimate whether the materials to be learned can be 
acceptably absorbed in a single session or exposure or whether much practice 
Will be required. It should be borne in mind that when only a few repetitions 
are needed and a large number of students must be taught, a communication 
method, like the lecture, may be quite efficient for presentation of the material. 
If more repetitions are needed to achieve the necessary level of performance, 
additional training techniques and devices become necessary. Some major 


Varieties of these are described in the next section. 


Training Devices in Individual Training 


A training device, as the term is used in this chapter, is any piece or 
group of materials or equipment used to train individuals. This general term 
includes two-dimensional displays of verbal, symbolic, or pictorial materials 
(such as wall charts, photographs, slides, transparencies, and motion pictures), 
audio signals which may or may not be associated with displays (such as vocal 
directions associated with a video display of parts being assembled), as well 
as real or simulated displays and controls which trainees can operate in order 


to develop skills or to learn principles. 


Perhaps the most widely used kinds of training devices 
use they are conceived as having the function 
l information presented by a teacher. A great 


Training aids. 
are training aids, so called beca 
Of aiding the exposition of verba 


346 - Training Programs and Devices 


many different types of training aids have been used to supplement lecture 
materials in teaching concepts, information, and simple skills. Some of the 
major types of these classroom training aids are described briefly here. A 


more detailed discussion and evaluation of these devices will be found in 
Lumsdaine (1960a, 1960b). 


Wall charts are for group display. Most commonly, they are photographs 
or schematic representations of system components, their connections, and 


their operation. Wall charts are easily produced, visible to all group members, 
and may be readily transported. 


Transparencies and slides are drawings or photographs on transparent 
material through which light is passed for projection of the picture onto a 
screen. They can be easily projected with one of several standard projectors. 


The use of color, multiple overlays (additional transparencies placed over 
another transparency to add additional informatio 


à c i n to the basic drawing), and 
masks removed at appropriate times to reveal additional content of a drawing 
give these techniques a reasonable amount of flexibility. Transparencies are 
convenient to use and are often cheaper and faster to produce than many other 


; OF a page i ie 
jectors are available for this purp Page Mà book, So-called opaque gne 
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the classroom instructor, dismantling or assembling the equipment and point- 
ing out the parts. In another, the instructor may be with the equipment, 
separate from the class, and instruct from there. In either of these cases, 
television receivers conveniently placed around the classroom allow all stu- 
dents to see the screens. Closed-circuit television is financially feasible only 
when large numbers of students are to be taught a particular subject or a 
series of subjects. 

Open-circuit television as a training technique should be mentioned, since 
its use in education is growing rapidly. Its values are similar to those of closed- 
circuit television and of motion pictures, but its application to operator train- 
ing for performance in systems has not as yet been undertaken. 


Cutaway mock-ups are generally made of actual equipment, partially cut 
away to show internal appearance and relationships. A cutaway engine used 
for instruction is shown in Figure 10.1. The parts in this engine move by 
power from a supplementary electric motor. Such cutaway models are of value 
in giving a group of trainees an over-all view of a piece of equipment, showing 
how its parts fit together and how the parts work together. Unfortunately, it is 
often difficult for all members of a class to see at one time the parts being 
pointed out by the instructor. When the cost of a cutaway is considered, its 
value for detailed group instruction in comparison with other teaching tech- 


niques is low. 


ctual equipment components so that 
they function much as they do when they are installed. These devices (such as 
the hydraulic-brake system of an automobile) show the actual functioning of 
à system. When they can be used to give instruction. in troubleshooting and 
maintenance in addition to their use for demonstration purposes, operating 
mock-ups have additional value. However, the difficulty of maintaining and 
Storing them, the infrequency with which they are used, and their cost combine 


to reduce their attractiveness. 


Operating mock-ups interconnect a 


connect actual equipment, or parts which 
These mock-ups are useful for lecture 
a general orientation to the device, 
lationships. Such mock-ups can, if 
nes if they are made of plastic or 
Iso be less expensive than opera- 


Nonoperational mock-ups inter 
Closely simulate the actual components. 
Presentations when the objective is to give 
its component parts, and their physical re 
desired, be made lighter than operational o 
Some similar material. These replicas can a 
tional mock-ups when enough copies are produced. 


/s represent components of the system hardware 


. ^nimated-panel display Der 1 
Pletorially or in the form of simple, semi-functioning models; sometimes both 


Methods are combined. One form of an animated-panel display is shown in 
igure 10.2. Essentially, this display is a large diagrammatic representation of 
à System, with simplified representations of the units within the system. Such 
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P of an auto 


* (Courtesy of the Office of Naval 
Training Device Center.) 


Mobile engine used to 


The training ai "Y Considerably in their value as aids 
in the teaching process. Probabty each of them has a place in this process 
depending on the intent and content of the course being taught. However, 
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before such an aid— particularly an elaborate one—is 
use, careful consideration should be given to the 
relevance of what can be taught by the aid, the effectiv 
the number of students to be taught, the time require 
change, ease of storage and access, and general flexib 


developed for classroom 
training objectives, the 
eness of the aid, its cost, 
d for its construction or 
ility in its use. 


Individual Trainers. When more 
the trainee to learn the knowledge or 
group presentation are helpful but u 


than several trials will be required for 
skill required, training aids designed for 
sually insufficient, In order to perform 


tor will need to learn to identify many 
E as many facts 


nceived a number of ad 

years ago, a 
Work was done 9n them at that time (Presse 1932. 
eir potential, ang thei i n 


ic, laser, 1960), Automated 
: Parned in a fixed sequence; à 
me of the NE beföre the next part of the material 
8 a learning segu s Phisticateq devices, the order of the 
S ca quence according to the ad of the 

n be kep € adequacy 


to 3 A 
the instructors use; Machines of be cde Students performance, for his or 
; e a : 
content material can be Presented for different Maas E TIBIE, Ae 
S of training. One s 


device which has considerable flexibility is s dé. 
devices can afford a wide range of learning ex own in Figure 10.3, These 


the more routine learning activities (such as ee they can be used for 
bols, or words) as well as for the learning f P. entification of parts, one 
principles. ore advanced concepts an 


correct response is re 
is presented to him. In SO; 
material is varied durin 
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1. SELECTOR BUTTONS 
f 2. SELECTOR INDICATOR 
f 3. VIEW. BUTTON 
4. MOTION BUTTON 
5, ON-OFF SWITCH 
6. FOCUS CONTROL 
7. VIEWING SCREEN 
8, HOOD 
9. RECORDER TAPE VIEW WINDOW 
10. RECORDER TAPE AND TAKE-UP 
then MECHANISM 
Figure 10.3. An automated teaching device, the AutoTutor ( 
his device is described by Crowder, N. A., pp E 


marked name). (TI 
in Lumsdaine and Glaser, 1960. Courtesy of Western Design and Elect 
ectron- 


ics, Division of U. S. Industries, Inc.) 


and Brainerd (1959) with the computer program controlling the ord 
type of problems presented to the individual student, the Eta er and 
answers, the knowledge of results, and the modification of the pre ing of the 
the material if the student made an error. Coulson and Silberman CIO of 
E the use of a flexible computerized teaching machine, desi a 
search, to study the principles and proccdures of value 3h B 
aching 
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technique. This research is also being extended to include the possibilities of 
e à 


using such a technique for multistudent classroom teaching, as well as for 
school administrative functions. 


Concept trainers are used when the concepts to be learned are too com- 
plex to be absorbed in a few trials from verbal descriptions, and when the 
principles to be used in task performance can best be simulated by physical 
objects and real actions. A good example of this type of learning is the case of 
the maintenance technician who must find malfunctions in complex electronic 
equipment and repair or replace the parts. He must learn a set of rules and 


have understandings which are recalled and applied when he is faced with 
instrument readings, wave forms, or other 


He must be able to trace the flow of elec 
isolate the malfunctioning circuit and replace 

Such a concept-training device is shown 
is a miniature representation of the componen 
a complex electronic system. After 


physical stimuli from the equipment. 
tricity from the instrument readings, 
the faulty unit. 

in Figure 10.4. The front panel 
1s, controls, and check points of 
à typical malfunction is set up in the system 


Figure 10.4. A concept trainer for + 


d utilization of appropr 
an airborne bombing-navigation system. 
described by French and Martin, 1957.) 


raining electronic technicians in 


iate conce, 


Pts for troubleshooting 
(From 


Lumsdaine, 1 960b; as 
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with one of 200 individual switches on the side panel, the trainee isolates the 
source of this malfunction. An instructor may coach the trainee, also guiding 
him away from illogical or unnecessary checks. In this way the student tech- 
nician can practice on a variety of equipment-malfunction problems of graded 
difficulty. With this type of trainer, the student learns the complex concepts of 
System functioning in direct relation to the stimuli which arise from the 
Physical equipment. The device is convenient to use for practice and is rela- 
tively inexpensive compared to the cost of using operational equipment. This 
type of training device thus eliminates the need for spending time in disassem- 
bling complex parts in order to diagnose routine malfunctions. The concept 
trainer allows the student to devote most of his time to learning those things 
that are most difficult to learn (French and Martin, 1957). 


Games embody a technique of training which is being applied in a 
Variety of ways. Games have been used most widely for training in the areas 
Of decision making and planning. For instance, the American Management 
Association's widely publicized top-management game is played as part of a 
Course in decision making offered by the Association to business executives 
at the AMA facility at Saranac Lake, New York (Ricciardi et al., 1958). The 
game allows the players to practice making and implementing business deci- 
sions in a simulated environment. Participants are divided into five teams or 
companies, each made up of three to five persons. These companies compete 
against each other in a hypothetical market with only a single product. The 
game is divided into periods each representing a quarter of a year; the five 
Companies generally play for 20 to 40 periods. Following each period, the 
AMA staff takes the decisions made by the teams; with the help of a computer 
and mathematical formulas representing a simplified model of the business 
World, the complex interactions among companies are quickly worked out. 
The resulting information on its own business position at the end of each 
Period is given to each company, as well as limited information on its com- 
Petitors, This information and past history govern the decisions which must be 
Made in the next period. The AMA game (and others which resemble it) 
Contributes to the learning process by requiring each player to take part in 
decision making and in the discussion and analysis of play following each run. 


Skill trainers are used for the many system tasks which demand highly 
developed perceptual-motor skills. It may be difficult to arrange for adequate 
Or frequent enough practice on the actual task to develop and maintain the 
Necessary level of performance on such skills. In other cases, it may be difficult 
us the operator to practice in the actual system without interfering seriously 
With the system's normal operations. Skill trainers typically provide conditions 
9t Practice which let the student know what results he has achieved. This 
Characteristic is of particular importance sirce it is only by knowledge of 
Tésults that the student can improve his performance. 
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An example of a skill trainer is a device which enables operators to attain 
skill in receiving code. Several types of these devices have been developed. A 
United States Navy device, called the Code Practice Keyer, can automatically 
transmit code from 10 to 30 words per minute. The Operator can practice 
receiving code through head phones or a loud speaker or by means of a 
blinker light. Such a device has been very successful in training individuals 
(singly or in groups) to develop individual proficiency in receiving code. 


Procedure trainers are mock-u 
place on which the trainee can 


k. For instance, in a cockpit procedure trainer 
be given training in normal and 
ine, the fuel system, and several 
» training in procedures for take-off, landing, ditching, 


d cockpit those instruments, con- 


i cen used not only for establishing initial per 
ormances, but also in the transition training of skilled pilots from one type 9 
Single-place aircraft to another. In the | 


Simulators integrate the various concepts and skills which an individual 
has learned into a complex oi near. 


total job situation a it him t° 
Pd : a nd permit him 
develop and maintain Proficiency. It is not always feasible fòr the trainee tO 
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practice such integration of skills in the actual job situation for reasons of 
safety, cost, or interference with system performance. A simulator is a special 
class of training device which attempts to reproduce or simulate the essential 
tasks of an operator or operators. In general it has a close physical resem- 
blance to the operational equipment, particularly with respect to the display, 
the controls, and the way one affects the other when in operation (Gagné, 
1954). In fact, some simulators use the operational gear; simulated inputs are 
fed into the displays so that trainees can carry out the normal operations and 
actions. Simulators range from those which present an individual operator's 
task to those which are used to train entire crews. In this section, only those 
Which are used for individual training are described. 

Some of the most highly developed simulators are used in the military 
Services. Flight simulators, in particular, have been widely developed for the 
Services and for major airlines. These accurately simulate such characteristics 
of the aircraft as the instrument panel, the indicator movements, the sound of 
the engines, the feel of the controls, and the effects of reactions made by the 
pilot. One use of such a simulator is in transition training—that is, the training 
required for a pilot who is proficient in flying one type of aircraft and must 
learn to fly another type with different characteristics. Figure 10.5 shows 
Such a flight simulator for the F-106, a United States Air Force aircraft. In this 


m 
i 
| 
I 


A flight simulator for the F-106 aircraft. The aerody- 
the cockpit instruments, 
the controls and instruments provide 
training of the pilot in flight, 
a flight “instructor and radar operator 
General Precision, Inc.) 


Figure 10.5. 
namic factors of actual flight are represented on 
and as the pilot operates the controls, 
realistic feedback to him. For integrated 
Communications, and radar phases, 
are necessary. (Courtesy of Link Division, 
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res in aircraft; yet 
andling these situations should they 
ts during and after each simulated 
Own actions, 


relatively easy, c 
them learned on the job. In some ca 


help from the instructor, 


onsideration should be given to having 
Ses, printed instructions will minimize 


LR e 
à § continuity by what he tells th 
trainee and by how he uses the devi must semis ne necessary set 
Tlier training to training with the device: 
tions of the physical equipment and envi- 
ut what is accurately simulated and what 


control over the functioning and use of 
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the device during the training situation. Most training devices have an instruc- 
tor’s console, which permits the instructor to control the training device and 
pace the material which he presents to the student. An instructor’s console 
usually provides controls by which the instructor can present normal, non- 
routine and emergency procedures or events to the student. The instructor may 
also be expected to provide the student with feedback on how he is doing, and 
to motivate the student by appropriate praise and reproof. The design of the 
instructor’s station and instructor's console requires considerable skill to make 
all of these responsibilities and duties possible of execution. 


Team Training 


is chapter, many adjectives are used in con- 


As pointed out earlier in th 
f these describe the nature or size of 


nection with the word “training”; some o 
the group being trained. Individual training and the kinds of programs and 
equipment relevant to it have already been discussed. In system terms, the 
typical form of individual training is component training, that is, the develop- 
ment of the skills necessary to permit the individual to “operate” his equip- 
ment so that the man-machine entity functions as a component of the system. 


At a higher level of complexity is a group of men interacting with 
Machines and with other men in such a way as to constitute a distinguishable 
h a group may man a bomber, the 


Portion of a total operating system. Suc i 
identification section within an air defense radar crew, a total radar unit, a 


ship (or a gun or some other equipment or function on that ship), a sub- 
marine, or any similar unit. This distinguishable portion of a larger system is 
Called a subsystem. When training is directed at the functioning of such a 
Subsystem, and particularly at the interactive procedures within the subsystem, 
One is dealing with what is often called crew training (or in system terms, 
Subsystem training). This is one kind of team training. 

. Finally, an additional kind of team training is what is called system train- 
ing; this, as its name implies, takes as its frame of reference the functioning of 
the total system, with particular emphasis on the interactive procedures that 
Obtain between subsystems (or in some cases between components) and the 
effects of these procedures on achievement of the goal by the system. These 
Subsystems perform related functions, they may be situated in separate geo- 
&raphical locations, and they may need to coordinate their work with that of 
a higher echelon which has to make decisions based upon information from 
these and other groups. This type of training is organized around the functions 
Of the system, although only part of the total system may undergo training. 
t selects for training a large enough segment of the man-machine system so 
that this segment can carry out all of its normal functions. It undertakes to 
train the subsystems simultaneously on an integrated, realistic problem, and 
Slves crews knowledge of results based on system performance. It requires 
Well-planned and observable situations for training. Examples of such situa- 
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tions are Army battle maneuvers with combat situations simulated in the gn 
realistic mock-ups of the critical operating positions ofa man-machine 2 ie 
for which personnel are to be trained, and operational set-ups in which "E 
personnel to be trained can work at their normal positions. Realistic but Ei 
lated inputs to the training environment are needed in each case, as well E 
a way to record team action and feed it back to the crews for their evaluatio . 
Usually, system training is undertaken at a time during the training 
sequence which is relatively late, so far as the use of hardware in the pei 
is concerned. Typically, it occurs at a time and under conditions in whic 
both individual training and (occasionally) crew training are assumed to have 
been satisfactorily completed. It is designed primarily to train the many crews 


comprising the system to perform functions which it is difficult, if not impos- 
sible, to practice in any other integrated manner. 


Training Programs and Equipment for Team Training 

Several types of team training are useful in, and fitted to, penes 
training situations. Whatever the form of team training, it can be greatly 
facilitated by the use of training d 


evices. Some of the programs and devices 
that are Particularly useful for team training are described here. 


emergency procedures, A fligh 


Perating conditions. Apart 
mplete safety, the simulato" 
and control in training. It makes po5 
tions. In addition, crew actions can be monitore 
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Figure 10.6. A simulator used for training the flight crew of a com- 
mercial iet aircraft. (Courtesy of United Air Lines.) 


building of its own and has a conning tower and bridge with normal operative 
equipment. The periscope rises through a floor on which an operator maneu- 
Vers five target-ship models at apparent speeds up to 50 knots. Attack condi- 
tions are realistically simulated. The submarine, under the control of the 
Students in the conning tower, is designed to respond normally to the helm. 
Periscope and radar presentations show depth changes; the periscope will not 
break surface when keel depth is more than 65'. A variety of submarine 
approach or attack problems may be run and evaluated with the Teacher. 
Possible problems include surface or submerged attacks on single fast or slow 
targets and on escorted or unescorted convoys; evasion of antisubmarine 
Vessels; long-range radar tracking and approach; and submerged sonar attacks. 
epeater instruments in a separate classroom allow additional students to 
follow the problem. One of the target ships can be made to seek out the 
Submarine with sonar in an attempt to sink it, thus training the submarine 
Srew in evasive tactics. 
Numerous other simulator devices and associated training programs for 
team training have been developed and used, and many others are under 
Construction, Several such devices and programs in use, selected for brief 
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"mes. 


Figure 10.7. A device for the 
crew personnel in certain System functi 
Research, U. S. Naval Training Device Cen 


integrated training of submarine 


ons. (Courtesy of Office of Naval 
ter.) 


NEWS) at the Naval War College (Davis & 
Tiedeman, 1960), and the Miniature Armor Battlefield (Baker, 1960) devel- 
oped by the United States Arm 


e Simulator is essentially an electronically 
controlled maneuver board for the simulation of naval warfare, used for the 
training of commanders and other 


participants, Commanders in combat infor 
mation centers equipped with radarscopes, Voice radios for communication 


with simulated forces under their command, and with othe 
in large-scale exercises and have an Opportunit 
of their operating procedures. 
large-scale war game on this s 


S 
r gear, move — 
y to evaluate the effectiyen 
NS are required to carry out 


miniature aggressor tanks. Each of the 


crew, the aggressor personnel sitting on an elevated platform at one end © 
the large room and the friendly tank-pla 
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steel platform at the other end. The movable platform is divided into five 
separate compartments, one for each three-man crew composed of a driver, 
gunner, and a tank commander. These crews communicate over a simulated 
radio network. The tanks are battery-powered, and their over-all movement, 
turret and gun movement (which are realistic), and simulated firing are radio 
controlled. The gun tube of each tank contains a small, bright light that 
projects a narrow beam, which when it strikes photoelectric cells on another 
tank, makes the *fired-upon" tank inoperable. Other devices for simulating 
mines, atomic weapons, and the like, are also employed. This Armor Battle- 
field device has the purpose of training crews to work together in many 
tactical aspects of operations. Use of this same simulator for proficiency 


Measurement is discussed in Chapter 12. 


Operational System Training. This kind of training is used in large, 
complex information-processing systems where information is collected, proc- 
essed, summarized, and displayed, and where decisions are made and actions 
taken. In such systems, decisions may be made at several levels, with the 
decisions at higher levels being based on highly summarized and interpreted 
information. Individuals placed in such systems usually have enough training 
to be able to perform their particular specialty to at least a minimally accept- 
able level. but generally have not exercised their knowledges and skills in the 
context of this particular system. In fact, the skills may be new ones which 
they have never used in any system. E 

A specific system training program which illustrates the principles and 
Operation of such a program is that in use by the Air Defense Command of 
the United States Air Force to train its Aircraft Control and Warning System 
(Biel, 1958). This program was first developed for the manual air-defense 
System. In this original system, all information-processing was done by human 
Operators. The simulation techniques of this program have more recently been 
modified for the Semi-Automatic Ground Environment (SAGE) system of air 
defense, which has largely replaced the manual system in the continental 
United States. In SAGE, electronic computers process much of the air-defense 
data and display it in integrated form so that human operators can monitor 
Many activities, make certain required decisions, and control many actions. 

Although this system training program has been used for exercising the 
entire North American Air Defense Command (Carter, 1960; Miller, 1960), 
its equipment and design are such that it is very profitably used at more 
frequent intervals for training by smaller units of the system. In the present 

Orth American Air Defense Command, these units are called regions, divi- 
Sions, and sectors. The earlier manual system organization was by forces and 

Visions. The program used is called the System Training Program even 

©ugh the units, by our definition, are subsystems. The major units, however, 
are large geographically and include many crews. 
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For simplicity in describing the System Training Program, a i npe ien 
force of the earlier manual air-defense system will be assumed as the uni 
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Figure 10.8. Map of a hypothetical Air Defense Force showing loca- 
tions of radar sites, Division Headquarters, Force H 
Headquarters, 


eadquarters, and NORAD 
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Aviation Agency centers, 
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shown on maps, and the like), which are used to simulate the informational 
inputs from those organizations or units with which the unit being trained has 
contacts or interactions. These surrounding organizations are referred to in 
this program as embedding organizations. For instance, flight plans associated 
with certain flights are called in normally from Federal Aviation Agency 
(FAA) installations in each division's area. In the System Training Program, 
these flight plans are computed in advance, tabulated, and sent as scripts with 
the film to each site. A trained individual, not a member of the crew being 
trained, simulates the FAA personnel’s role; at the times indicated on the 
flight-plan script, he telephones the plans to the crew being trained. 

Radar and other track information about aircraft penetrating the area of 
the force being trained is read from scripts and is called in over telephone or 
radio by selected crew personnel in other installations not participating in this 
training exercise. This information may be reported from radar sites in each 
of the adjacent forces, from airborne early-warning aircraft and from one or 
more picket ships, as shown in Figure 10.8. 

Other personnel not participating in the exercise as crew-member trainees 
simulate interceptor-base personnel and interceptor pilots. They are located in 
another room. When required, the air-defense crew being trained sends simu- 
lated interceptor aircraft to intercept and identify unidentified aircraft. The 
Tesponsible radar-crew members direct the simulated interceptor pilots, using 
Simulated radio channels, and these pilots operate generators that produce 
Simulated radar blips which indicate the location of the interceptors. The 
Benerators produce blips on all radar consoles in that site. These simulated 
Interceptors which appear as blips on the consoles are maneuvered as directed 
by the crew according to a set of realistic flight rules for simulation. The 
Simulated pilots have materials which provide them with detailed information 
9n each aircraft which they might be directed to intercept and identify. 

These integrated, simulated inputs are received and processed during the 
training exercise by crews in all the radar sites, at each division headquarters, 
and at force headquarters. Each crew trains in its normal environment using 
!'$ operational consoles, its plotting boards, and its telephone network between 
üir-defense radars and higher headquarters (but not to the FAA, interceptor 

üses, or airborne interceptor aircraft) and carries out its normal operating 
Procedures, At division and force headquarters, of course, additional simulated 
Mputs, such as intelligence reports, are put into the system. 

Observing the crew at each headquarters and at each radar site is the 
Training Operations Report (TOR) Team. This team has printed materials 
Which describe the details of the problem and on which records can be kept. 

hile the problem is being run, the team observes and records crew actions, 
Particularly those dealing with situations which have been identified in advance 
as important. The team verifies with adjacent radar site TOR Teams that 
Critical messages were sent by and to the crew it was observing. 
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As soon as possible after the exercise, the TOR —— ok sar 
vations to the crew or to the senior officer who, in turn, gives t : -— ies 
his crew. Crews in subsystems in different locations, working toget sie pins 
of the system, each get the over-all picture of system success and the > 
of their own performance. In addition, a debriefing or discussion oon eR 
held by the crew in each location to try to diagnose its problems as a pa next 
the system, and to propose solutions which might be tried during the 
training exercise. u oe 

The general intent in such a training program is for the participan . 
find out about the over-all system performance, their own subsystem pte 
ance, and to diagnose and solve their operational problems, particular Y 
methods of crew interaction. Subsystem performance and even inifarmanon 
from the crew personnel on individual actions must be used in the problem 
solving. This contributes to a feeling of group belongingness, group partic! 
pation, group problem orientation, understanding of procedures, understanding 
of the group’s problems, and a better understanding of what will be tried nex i 
In fact, in this concept of system training it is difficult to find an appropriate 
use for the term "instructor." Although TOR Team members and simulator 
personnel record the actions taken and know in detail what the simulation 


situation was, the officers and the airmen in the crew do most of the prob- 
lem solving. 


The cost of a simulation trainin 
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The same exercise can be repeat 
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PLANNING FOR THE DEVELOPMENT OF 
TRAINING PROGRAMS AND DEVICES 


[B] 


Planning for training programs includes planning for the development 
and use of training devices. It is closely related to, and should be an integral 
part of, system design and development of the system as a whole. 


Training Design in the System Framework 


Throughout all stages of system development, training requirements must 
be considered; as the details of the system are worked out, training plans must 
likewise become increasingly detailed. This is particularly essential in military 
Systems, because training is one of the system functions which has to be carried 
out during peacetime. In consequence, system training must be recognized as 
an important, often independent, mode of operation of the system. A program 
is usually needed to develop and maintain human proficiency, and the capa- 
bility for this type of training must be incorporated in the operational system 
and its equipment. Economy, quality, and timeliness of system training suggest 
the need to include facilities for training in the general operating requirements 
for the system, as well as in the operating system description. The operation 
of the training program can be worked out in more detail when the internal 
design of the system is prepared. In some military situations, only one inte- 
grated system is built and installed. To develop a high level of system per- 
formance, the system itself must be used for training. 

The designers of a system training program should be intimately familiar 
with the requirements and design of the system. Whenever possible they should 
Participate in the analysis and design process. This insures proper understand- 
ing of the goals of the system, its operational problems, information to be 
Processed in the system, decisions to be made, actions to be taken, and so on. 
Informed judgments can be made as to what must be simulated in the system 
for adequate training, the accuracy with which it must be simulated, and how 
It can be simulated. These judgments may have to be modified as system 
development proceeds. Furthermore, the training program must be planned 
to be flexible, so that it can change as the system and its operating procedures 
change after installation (see Chapter 11). An operational system training 
Program must also include plans for the production of integrated, simulated 
inputs, Since this is a major task, serious consideration needs to be given to 
how it will be achieved for an initial indoctrination training program, as well as 
Or exercising and training after the system has attained operational status. 
bs As soon as the component design is under way and operator tasks can 
Specified, a more detailed task analysis can be prepared for each operator 
de d. This not only makes it possible to pian the training objectives and 

ailed requirements for the individual operator training, but also facilitates 


Training Programs and Devices 


stored or memorized, what interactions are 
performing closely related functions, what TeSpi 


what limits of accuracy, and what skills 9r procedures are necessary to per- 
form the job. In other wo it i 


and to what level. 


ten for the training aids an 

cations must include the work- 
s and their ranges and tolerances, 
ontent, and in some cases, where 
CT Programming requirements fO! 


ing individual or system perform- 
ent with 


It is through study 
design and development, 
and development of th 


of a system and 


n e ë i i 3 
Its training re uirements early in Ít 
and through the taki n 


system if this is n 


iona 
das ence the design or the operation 
à ecessary, in order that an effective training capacity may b? 
achieved. In pract = 8 cap 


3ys been Possible to get potentia 


— William C. Biel = 367 


training needs recognized by system designers early enough in the design and 

development of systems to follow in sequence all the steps described in this 

Section, and to have appropriate training programs and devices ready when 
they are first needed. Such a situation is not satisfactory. When training plans 
are made late, it is clear that one must start from whatever point the system 
1s in its development or operational phase to improve the training capability. 
However, this delay does not relieve training designers of the responsibility of 
thoroughly understanding the system and its operator tasks before designing 
the necessary training programs and devices. 


Human Engineering and Training 


Another very important activity must also be carried on during the design 
phase, particularly at the time of allocation of tasks to equipment and opera- 
tors, as well as during the design of component hardware; it is that of design- 
ing the tasks and the equipment so that the human operator can perform his 
tasks efficiently. This, of course, must be done with consideration of what leads 
to optimal over-all system efficiency. This aspect of design activity is called 
human engineering and has been discussed in detail in Chapter 5. Good 
human engineering of equipment is of great importance to training, because it 
can simplify the training problem immensely. By making the displays easier 
to read and interpret, by simplifying the decisions required and the actions to 
be taken, and by contributing in many other areas where operator capabilities 
Must be considered, the amount and type of training can be greatly influenced 
in the direction of increased efficiency. In addition, it may be noted here that 
appropriate human engineering of equipment has major implications for the 
Selection of personnel to be trained and placed in system jobs, since good 
equipment design can bring about a reduction in the basic operator qualifica- 
tions required for these jobs (Taylor, 1960). 


Use of Simulation in the Design of Systems 


In the design and development of many large systems, it is advantageous 
tO set up a simulation of the system or its major parts, so that studies can be 
conducted and decisions reached about directions of development. Such 
simulation is useful in as early a phase as that of operational design. Hardware 
in Such a simulation probably would include critical simulated consoles with 
essential displays and with operable controls. Ideally, for complex systems a 
Computer is an integral part of the simulation. The computer can generate the 
Simulated environmental situation displays for the consoles or other apparatus 
according to a plan, simulate certain characteristics of the system, and respond 
to and record the responses of the crew members. 

Such a simulation, very general at first but more detailed as the design of 
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the system progresses, can be very useful in providing a vehicle E diei 

system designers and training developers in many ways. Some o 

D oc oe m the designers of the system, as well as those designing E 

components, to determine what operational problems sii d ed 

system and to study these experimentally. As components are emm Mem 

built, they can be substituted for mock-ups. Results, even those whic 

from trial and error, enable the designers to make better decisions. -— 
2. Knowledge is gained concerning appropriate allocation of tas 

human operators versus machines, or between 


. H is 
operators; more information ! 
gained regarding display, 


information load, and control problems, pent 
layout, operational procedures, potential capacity of the system, opera 
requirements for each job to be performed, and many other factors. — 

3. Experience gathered with simulation permits better determinatio jug 
the need for and contributes to the subsequent planning for individual MA 
devices; it also permits early decisions on the need for a system training P 
gram and associated equipment, and on the detailed nature of both. 


4. In later stages of a more detailed simulation, the setup may itself b» 
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other system simulation, before such a model is built and before such studies 
are conducted, a detailed operational analysis of the system to be simulated 
must be made if realistic or useful results are to be obtained. 


@ PSYCHOLOGICAL CONSIDERATIONS IN THE DESIGN 
OF TRAINING PROGRAMS AND DEVICES 


[t] 


There is a variety of factors, considerations, and principles which guide 
the training designer in the design and development of training programs and 
devices for complex man-machine systems. Some of these have been alluded 
to in previous sections of this chapter. In this section, they will be brought 
together and augmented by a few considerations which have not been made 
eXplicit earlier. For the most part, the emphasis in this section will be on the 
Psychological principles influencing the training designer's decisions. 


Human Problems in Complex Systems 


Underlying the design of training programs and devices for today’s com- 
plex man-machine systems must be an appreciation of the characteristics of 
Such systems, including the psychological and social factors which make 
Sophisticated training programs particularly necessary. Eckstein has discussed 
the characteristics of large-scale automated systems and their implications for 
training design (1959). As he points out, in today's computerized systems, 
the human operator perceives his significance relative to that of the machine 
as small and tends to feel loss of control over his working environment. His 
task may be narrowly constricted, as may be his knowledge of the total 
System and of the relationship between his task and the performance of the 
System. Operators may be isolated, sometimes in darkened rooms, with 
mechanized and formal communications, rather than enjoying natural face-to- 
face relationships. The operator receives little feedback on the effectiveness of 
his actions, and consequently his ability to improve his performance is limited. 
The effective training program must recognize these factors and provide spe- 
Cific measures within the training program for dealing with them. One antidote 
to these particular factors lies in system training, and especially in the provi- 
Sion of means for effective informational feedback in the context of group 
Problem-solving sessions. 

In addition to this appreciation of systems in general, the training designer 
Must, of course, become intimately familiar with the particular system for 
Which he is designing a training program or specifying a training device. He 
Must understand in detail such things as the system mission, system perform- 
Ance specifications, system characteristics and procedures, operational prob- 
Sms, and the system environment. With this background and intimate 

Nowledge of the task analyses of the operator positions, he is better able to 
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make intelligent decisions about what skills must be trained, to what level, and 
by what techniques. In addition, he will be prepared to determine what system 
characteristics need to be simulated, with what degree of realism; where 
simulated inputs should and can be introduced into the system; where perform- 
ance can be observed and recorded; and what the significant elements of 
system performance are which need to be reflected in post-exercise feedback. 


Training Objectives and Training Requirements 


The training designer must make explicit the objectives of the training 
program and devices to be developed. Generally, the ultimate goal of indi- 
vidual and team training is the same—— maximum performance of the man- 
machine system. Practically, however, the goal of individual training 15 
improvement in the performance of the individual to an adequate level in the 
shortest amount of time, whereas in team training the goal is more likely to be 
the achievement of maximum performance of the system, The determination 
of how best to attain these individual and system goals involves establishing 
the subordinate or intermedia 


1 ry training objectives, For example, in the case 
of a system training program, major intermediary Objectives include, among 


others, the development of system awareness and the development of system 
flexibility (see Chapters 11 and 13). S 


: ystem awareness is the ability of the 
individual in a system to understand the relationship between his task and the 
functioning of the over-all system. As di 


mit his knowleg tem. 
a : ge of the total sys 
On the other hand, if the system is to perform effectively, the operator must 
develop an understanding of the interr à 
System and of the effects of his action 


S, to th i ems 
and to the optimum functioning of the over. yan. dl Gf these: proble 
A second intermediary training objective, system flexibility, refers to the 
ability of the system to adapt to change, particularly to sudden and unantici- 
pated circumstances. An effective training program Promotes system flexibility 
through simulation of a wide range of conceivable conditions, through insuring 
that the personnel become accustomed to dealing with unexpected contin- 
gencies, and through providing experience in the development and use of 
alternate procedures. 
Where a training program:or device has multiple Objectives, some of thes? 
may not be adequately served by the training instrument unless they are 
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explicitly formulated. For example, a system training program must often 
serve as a vehicle for system diagnosis and revision as well as for system 
exercising, and therefore will require not only means by which existing proce- 
dures can be practiced but must also provide means for developing, testing, 
evaluating, and revising procedures. The training designer thus bases his 
statement of training requirements on explicit training objectives. He specifies 
the information, skills, and attitudes required by operators at each position in 
the system. These specifications also describe the level of proficiency needed 
to perform each job satisfactorily. Techniques for performing the analyses 
leading to such specifications have been described in Chapter 6. 

From these requirements it is possible to make decisions concerning 
training content, methods, and devices, and it is also possible to develop the 
Proficiency measures and standards used to evaluate individual and team 
Performance. By using these measures during a training exercise, the training 
designer can establish how well the training objectives have been met; with 
these measures he can identify those parts of the program needing content or 
method improvement. Further, he can plan for the providing of guidance to 
individual operators concerning their proficiency. 


Training Content 


From the training objectives and requirements, the curriculum for the 
training program is developed. The three major parts of the curriculum are 
content, training methods, and training devices. Content includes the material 
to be presented, its organization, and its sequence. When planning the material 
to be presented, consideration is given to the previous experience, capabilities, 
and other characteristics of the trainees and to the training requirements of the 
tasks, These considerations determine what is to be taught, the level at which 
it is presented, the organization of the material, and the sequence in which the 
Material is presented. 

In addition to the content which is taught, early emphasis should be 
Placed in the training on an overview of the system and its mission, with 
repeated clarification of the relationships of the tasks to the larger functions 
Of the system. This contributes to the student's understanding of the purpose 


Of his job in the system. Students need to be convinced that what they learn 
d to the system. Only with such motivation can 


9r do is important to them an ) : 
s will learn their tasks to the 


it be expected that a high percentage of student 
Tequired level of proficiency. 


Training Methods and Devices 


The training program designer must decide whether the content to be 
taught and the required level of operator performance can best be achieved 
Y individual training through classroom techniques and individual training 
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nations of these. Some tasks will require skills and information whic cim 
learned separately and therefore imply individual or component iem 
others require the integration of component skills through subsystem. tra Ed 
and still others require an appreciation of the operational relationship es : 
components or subsystems, and therefore imply system training. In genera fof 
minimum level of component performance is necessary, but not sufficient, 
effective subsystem performance; similarly, 
performance are necessary, 
Selection of training 
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situations on which training is given must simulate real life. In the first case, 
decisions need to be made concerning what critical tasks the operator must 
perform and what essential information is needed for the performance of these 
tasks. This leads to a concern with what instruments are required to display 
this information and with what precision they must display it. As a general 
principle, information of this type should be displayed in as realistic a fashion 
as possible,’ although it may be appropriate for noncritical information to be 
displayed using simulation which is less accurate, or perhaps only to be 
represented by dummy instruments, or not to be displayed at all. In the second 
case—that is, fidelity of simulation of the stimulus situation—it is generally 
essential that the situations or "problems" which are used for training be 
realistic and show appropriate relationships between critical variables. These 
Problems should be planned so that the operator can be exposed to a wide 
Variety of situations of varying complexity which he might face in real life, 
ranging from normal through emergency. This is important in order to broaden 
the experience base of the operator and to cultivate flexibility and adaptability 
on the part of the individual or system. The proper sequencing of these train- 
ing situations is likewise an important consideration in order to obtain maxi- 
mum training value. 

In most cases, the fundamental problem with respect to either of these 
two aspects of simulation (though it relates more clearly to the case of equip- 
ment and functions than to stimulus situations) is to optimize the relationship 
between fidelity, transfer, and cost. This relationship for the design of a 
training device has been shown schematically in Figure 10.9. Basically, the 
training designer hopes to be able to locate as accurately as possible the 
àmount of fidelity of simulation required to obtain large amounts of transfer 
Of training at a point where additional increments of transfer are not worth 
the added costs. This amount of fidelity is a point beyond which, if fidelity is 
increased, the transfer may increase, but not proportionately to the cost; in 
Other words, it is a point of diminishing returns. For many reasons, this 
O9Ptimization is a complex problem. 

The relationship between degree of fidelity and amount of transfer is not 
For example, a number of studies have been reported 


f simulators to a single type of aircraft 
esented the aircraft characteristics 


Clear and unequivocal. 
which transfer from several types o 
Was investigated, where the simulators repr 
With varying degrees of fidelity. Some of the studies report a significant rela- 
tionship between degree of fidelity and transfer of training (for example, 
Dougherty, Houston and Nicklas, 1957; Ornstein, Nichols, and Flexman, 
1954). while others do not (for example, Mahler and Bennett, 1950; Wil- 
Coxon, Davy, and Webster, 1954). The inconsistent findings are difficult to 
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In weighing the over-all relationship between cost and transfer in flight 
simulators, and perhaps in the case of some other simulators, one important 
point to consider is the goal or goals of the training. For example, there may 
be no opportunity for the live practice of certain operational flight tasks before 
the occasion arises when they must be performed. This is particularly true in 
the case of emergency procedures. When the occasion does arise, perfect 
execution may be required the first time. Thus, with respect to such tasks, it is 
imperative that simulator training be designed to meet the requirement that 
there be maximum transfer to the live situation on the first trial. 


Knowledge of Results 


Knowledge of results (KOR) provides learners with information on how 
Well they are doing and whether or not their responses are correct. Laboratory 
Studies consistently report either that KOR is essential for learning to take 
Place, or at least that KOR produces more learning or more rapid learning 
than performance without feedback. Wolfle (1951) cites a number of studies 
indicating these consistent findings. Despite the extensive laboratory investi- 
Bations of KOR, however, this factor deserves extremely careful attention 
When a training program or device is being planned and warrants a great deal 
More empirical study and field observation. There are many variables com- 
Prising KOR, including timing, type, amount of interpretation, relevance, 
accuracy, motivational value, individual versus team presentation, and other 
Utilization variables. While many of these dimensions have been studied at 
length, investigation of their interactions is an area where the surface has 
hardly been scratched. Undoubtedly, for example, under one set of circum- 
Stances, one technique for presenting KOR is more effective than another, 
While under different circumstances (involving perhaps a different task or 
altered timing of KOR) the same relationship does not prevail. 

Recognizing the complexity of the area, the multitude of studies, and the 
fact that much investigation has yet to be done, it is nevertheless possible to 
formulate some generalizations representing today's state of knowledge about 
KOR. This may ‘serve as a useful frame of reference for the training designer. 
In this section such a set of generalizations will be listed, along with some 

rief comments. The generalizations relate principally to individual as opposed 
- team KOR, since most of the reported research has been done in connection 
With individual learning. 

KOR increases the rate and level of learning. As discussed above, the 
*Xperimental literature is consistent in the contention that KOR results in 
More learning or better performance, or at least faster learning, than perform- 
ance without feedback. It is questionable, in fact, whether learning occurs 
Without it, This contention is essentially universal, regardless of whether the 
servations deal with individual or group training, simple or complex mate- 
"al, motor or verbal tasks. 
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Part, the generalizations discussed above, derived from investigations of KOR 
in the individual learning situation, do appear to apply to team KOR as well, 
although there has been very little empirical study of KOR on the team level. 
There are, however, important differences between individual and team KOR. 
Individual KOR refers to knowledge of individual performance which a person 
receives about his own behavior in performing a task individually or as a 
member of a group. Team KOR, in contrast, refers to the feedback which the 
Sroup receives about team (or system) performance, goals, organization prob- 
lems, and the like, and traditionally deals with individual performance only 
(if at all) as it relates to group goals. The variables discussed above require 
empirical investigation in the team context. In addition, those factors which 
are introduced, or greatly intensified, by virtue of the existence of a team, 
Tequire appreciation and further study. Consider, for example, experience 
With respect to one aspect of KOR in the Air Defense Command system train- 
ing Program. In this program, the postexercise debriefings are viewed essen- 
tially as group problem-solving sessions. Detailed, objective KOR provides the 
Stimulus for the group to identify system problems and formulate new solu- 
tions to be tried out in subsequent exercises or live operations. In this situa- 
tion, it is important that a climate be created in the debriefings in which 
exercise observers, debriefing leaders, and crew members can communicate 
freely, In military settings, where the traditional hierarchy may tend to inhibit 
free communication, this issue calls for special attention and study. 


Frequency of Training 


When plans are made to employ training devices, the question of how 
Often they should be used must be answered. No general principle can answer 
this question for all training devices, since optimal frequency varies with the 
training Objectives, the tasks, the device, the trainee(s), the quality of the 
Instructors. the length of each exercise, and many other factors. The best 
answers can be obtained empirically from experience with the device. From a 
Practica] point of view, training devices have generally been used too little 
rather than too much. However, frequency is important mainly because it 
Provides opportunity for other factors in training, such as knowledge of results, 
to have their effects. Usually, frequent practice is essential for learning. Per- 

aps it may best be said that the optimum frequency is that which best 
achieves the training objectives in the shortest time at the least cost. 


Evaluation and the Criterion Problem 


When a training program or device has been developed, it is desirable to 
Subject it to some sort of formal evaluation, in order to assess its effectiveness 
and in Order to provide one means of determining requirements for its modi- 

Cation. The fundamental criterion for evaluating a simulation-based training 
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program or device is the extent of transfer of training to the live situation. 
Thus, for example, a number of empirical studies—such as Ornstein, Nichols, 
and Flexman (1954), Mahler and Bennett (1950), and Wilcoxon, Davy, 
and Webster (1954)—have investigated transfer from flight-simulator training 
to flight performance in the actual aircraft. In these studies, experimental 
groups received a period of training on the flight simulators, where they 
practiced basic flight maneuvers such as climbs and turns. Their training was 
then continued in the actual aircraft. The control groups received equivalent 
total training time, but entirely in the aircraft. Flight proficiency of the experi- 
mental and control groups was compared by means of such measures as 
ratings on instrument maneuvers, accident rates, flight failures, and number 
of additional flights needed. The experimenters concluded that transfer of 
training took place; and in fact, that those trained initially on the simulators 
met the flight proficiency standards earlier than did those who had training 
only in the aircraft. 
à Many training devices and programs have not been validated to this 
mr s bed instances they cannot be, since rigorous evaluation would 
q at they be validated against the ultimate criterion—that is, perform- 
rte the individual or system in the live situation under the conditions for 
row d DRE ea this ultimate criterion is often wan 
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back or debriefing. Two crews (experimental) received practice, knowledge 
of results, and debriefings, while two crews (control) received the same num- 
ber of training exercises but no knowledge of results or debriefings. All crews 
were given the same test-retest and training problems. The experimental crews 
showed a consistent and large advantage on the various performance measures, 
which included such items as detecting flights, establishing tracks, maintaining 
continuity on critical or important flights, telling tracks to adjacent sites, and 
taking appropriate tactical action on critical flights." This study suggests how 
effectiveness may be achieved with this particular type of training program, 
but more importantly, it illustrates one of the methods by which a training 
technique may be validated using intermediate criteria where the ultimate 
Criterion is not available. 


System Training Principles 


Although the detailed characteristics of a training program are deter- 
mined by the particular system which is to be trained, there are a number of 
general principles of operational system training which must be considered in 
designing such a program for any complex man-machine system. They have 
been established through experience in the analysis, development, and training 
Of various systems, both in the laboratory and in the field. For the most part, 
they parallel principles which are used in individual training, but with some 
Modifications and shifts in emphasis. The focus in the examples that follow 
is not on learning theory, but is on the conditions which must be established 
in a training program to optimize learning. The focus, further, is on the 
Acquisition of complex rather than simple skills. 


Promoting System Skills and Understanding. For complex systems to 
Perform effectively, it is not sufficient for operators to be proficient at their 
individual tasks. Each operator must understand the relationship of his task 
to the functioning of the system, and must develop skills where necessary for 
interaction with other components in the system. Thus, an important feature 
Of system training is the design of simulated system inputs planned in such a 
Way as to require not only component skills but also a high degree of goal- 
Oriented interaction among the operators in the system. Such, for example, 
Would appear in situations calling for load balancing or load sharing, estab- 
lishing priorities and procedures, or initiating the use of alternate facilities. 

he development of these skills implies a training vehicle which permits the 
Operator to learn, not only what his task is and how to perform it, but also 


* Also of interest in this study is the fact that the performance disparity between 
€Xperimental and control crews was particularly great im the case of functions which 
Were relatively “invisible.” The visibility of a function meant the extent to which its 
Per Ormance could be observed or its consequences determined by the individual or 
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what its purpose is, how the task relates to the goals of the system, and what 
the "environment" is in which the task is performed. "—— 
Understanding the environment means that he must rir. duin 
organization within which he operates, including its formal and ime : in 
ture; and the resources and communication channels which he can uti iz de 
solving problems related to his tasks. He must understand the finen vs 
capabilities, and limitations of the hardware components of the yen s 
most understand the Auman environment, including the functions and abili i : 
of the people in the System with whom he interacts, and the relationship ee 
their roles to his own. This understanding is gained partly by orientation an 
individual training, partly through participation in a variety of carefully 


designed system training exercises, and partly by means of postexercise group 
discussion and problem-solving sessions. 


Encouraging Goal-Oriented Exploration. 
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and it is not necessary to simulate all the various agencies “surrounding” each 
unit, as is done when separate, smaller-scale exercises are conducted. This 
Tesults in a greater degree of realism, achieves more complete training, and 
affords a more valid indication of system effectiveness. However, it may be 
noted that other embedding organizations around the larger unit being trained 
still need to be simulated. 

Completely simulated exercises of immense scope have been successfully 
conducted, such as those designed for the entire North American Air Defense 
Command (NORAD) (Carter, 1960; Miller. 1960). However, it is not prac- 
tical to run exercises of this scope frequently. Special types of training must 
be provided for various sizes of subsystems to meet a variety of training needs. 
The training designer must plan carefully to insure a training vehicle which is 
adaptable to many different exercise configurations and system sizes, the 
Selection of these being dictated by training considerations, such as current 
Operator skill level, as well as by such practical considerations as scheduling 
Problems, cost, and the maintenance of an acceptable level of live operations. 


Simulating the Environment Adequately. Although exact simulation is 
Neither feasible nor necessary, an effective system training program must attain 
àn adequate degree of realism. An attempt is made to simulate all inputs 
Which significantly influence decisions and actions, doing so with enough 
realism that no important differences exist between simulated inputs and pre- 
Sumed live inputs. All significant interactions within the system, and outputs 
from the system, must also be able to take place realistically. And the system 
Must be made to suffer the consequences of its own actions in a realistic 
fashion, Thus, in an air-defense exercise, if the system fails to defend an 
important target area which includes an air base, that base would be lost to 
the System. Adequate simulation is essential to insure that skills acquired 
during training will transfer to live operations. Realism also helps to stimulate 
the interest of the trainees, which in turn facilitates the acquisition of the 
Necessary knowledge and skills. The degree of realism implied by this principle 
necessitates gathering large amounts of data about the live operations environ- 
ment of the system to be trained, both with respect to normal day-to-day 
OPerations and potential emergency situations, and subsequently integrating 
these data into adequate simulation of the real environment. 


Stressing the System. A simulation-based system training program 
Should be so designed that the designer of specific exercises should essentially 
ave complete control over the nature and quantity of inputs to the system 
cing trained, He then can design problems to stress the system in accordance 
With Whatever the training needs dictate, as well as to "test the limits" for 
Purposes of evaluating system capability. Thus, training problems can be 
signed to stress those system functions which are deficient or those parts of 
© system which must perform particularly critical functions. Most important, 
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the system can be confronted with stressful problem situations it would x 
encounter in normal daily operations. Many systems have no opportunity ia 
experience in live operations anything resembling the conditions He Mic 
prevail were they called upon to perform their ultimate mission. For exam ue 
in the case of a military system, the exercise designer can introduce heavi 


s x i the 
than-normal input loads, enemy attacks, and various kinds of attrition to 
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raining Program as an integral part of contin- 
uous system operations. Thus, the training program is operated on-site, rather 


than in a central training facility; it is conducted in the operational setting, 
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using the normal operational equipment and communications channels insofar 
as possible, rather than some form of simulators or mock-ups; it is an ongoing 
program rather than being limited to training for initial proficiency, with 
exercises conducted frequently throughout the life of the system, to insure 
that system potential is realized as system functions and characteristics evolve 
and change over time (and as personnel change through rotation or turn- 
Over); and it is supported by management or command level personnel as a 
regular and essential part of system operations. 
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PREVIOUS CHAPTERS HAVE DESCRIBED THE 
techniques used in defining human tasks, in providing 
individual training to insure adequate levels of skill, and 
in measuring the performance of human beings in carry- 
ing out these essential portions of system operations. 
But there is an additional and highly important aspect 
of training that reaches beyond the capabilities pos- 
sessed by the individual man-machine combination. This is the process of 
team training, the requirement for which arises out of the many and varied 
interactions that characterize modern, highly complex systems in an ever- 
changing environment. Many operating functions of such systems and their 
subsystems are performed by teams, made up of people and machines whose 


activities must mutually support and modify each other in a coordinated 
fashion in the attainment of common goals. 


The successful operation of teams in the 


accomplishment of system pur- 
umption that individual skills 
men have already been learned. 


: 1 m. It is possible for some of these 
working procedures to be specified as established functions. which can be 


Oyed to optimize team procedures 
1 method of primary importance to 
this effort. Here are described a number of kinds of changes in work proce- 
dures which can result from team trai ing. They include such factors as 
dencies of individual actions, analy- 


al forms of adjustment to load and 
work flow. For psychologists interested in training, i x some 


such modification in human 


how such team training effects can be 


amework of system exercises, and some suggestions are 


made of the types of concepts required to describe the results 
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TEAM FUNCTIONS AND 
TRAINING 


Robert Boguslaw 
and Elias H. Porter 


Generally speaking, the term team is used to describe a collection of 
human individuals who work together to achieve a common goal. We are 
familiar with football teams, basketball teams, and even combat teams. A new 
type of team is emerging in the enormously complex, computer-based systems 
of today. This type of team requires new perspectives. 

To understand the operation of large systems, it is necessary to consider 
something more than the attributes of the human beings working within it. 
Machines, computer programs, and, more generally, programs of interaction 
which can be adopted to achieve a system goal are all involved in the analysis 
of team behavior. Of particular concern in the examination of this behavior 
is its relation to the attainment of goals and some of the ways it may be altered 
to achieve greater effectiveness. As Forrester (1958) emphasizes in his 
account of industrial dynamics, the various parts of a total industry, although 
apparently independent, are markedly interdependent. The responses of re- 
tailers, distributors, and wholesalers can have the cumulative effect of over- 
loading a manufacturer or causing layoffs, even though the rate of sales by 
the retailers remains fairly constant. Mail-order houses are plagued by prob- 
lems incident to the apparently simple process of filling orders. The launching 
of missiles involves the development of highly complex procedures by mem- 
bers of a team whose members may be oaly dimly aware of each other’s 
existence. A nurse finds that providing services in a hospital depends upon the 
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effective cooperation of many people, upon equipment, 
administrative arrangements as well. 

While it is possible to have a team perform its goal-directed activities by 
following fixed procedures, the programming of which is under the control of 
external instructions or of previous training, this c 
special conditions, which we shall discuss subsequentl 
of complex man-machine systems requires that prov. 
for the development of individual skills but for the 
manner which will insure the 
flexible work procedures. 


and upon a varicty of 


an be done only under 
y. The effective operation 
ision be made not simply 
adaptation of a team, in a 
acquisition and progressive modification of 


5j TEAMS, TEAM FUNCTIONS, AND TRAINING 


The Meaning of Team 


t: jit is necessary to differentiate the 
"Si" a A group,” “small group,” 
social system,” and “society.” These latter ter; 


Simple or complex, close or 
relationship among members 
however, to use the term “team” to 
P among people. It ; i 
a relationship in which People generate te) m Mee ii Wm = ien 
Possible their interactions with machines, machine s Ue iie or 
People in the pursuit of System objectives, Procedures, and other 
Teams are contrived by me 


n to accomplish certai 
j ain 
ships among men, work procedu ee 


i eer The relation- 
res, machines, ang machine pr, 


ocedures have 


—Robert Boguslaw and Elias H. Porter - 389 


meaning for the team only to the extent that they contribute to or detract from 
the effective and reliable attainment of the goals. 

What does this imply for the system developer? In his conceptualizations 
of systems he must include work procedures, machines, and machine proce- 
dures (such as computer programs) as "members" of the teams with which 
he is concerned. The rigidities and flexibilities of procedures, equipment, pro- 
grams, and their interaction with each other will do much to influence the 
effective and reliable attainment of goals. Further, the system designer must 
be prepared to take into account the impact on the entire team of a change in 
any one of these members. 

To illustrate, let us consider a very simple, yet not necessarily obvious, 
equipment change which altered team functioning. The old-fashioned ball of 
American football used to be much thicker than the football of today. The ball 
was made slimmer to make it easier to pass with accuracy. Obviously, this new 
dimension markedly altered the tasks to be performed by most team members. 
Defense against passing required skills which were clearly different from 
defense against a running attack. Moreover, the increased complexity of the 
game demanded emphasis upon the higher-order skill of flexible "program- 
ming." Players today must be faster in shifting their defense (or their offense) 
in response to the particular way in which a preprogrammed play in fact is 
developing. 

In the RAND Corporation studies of air-defense crews (Chapman et al., 
1959) it was found that the ability of the crews to alter their work procedures 
resulted in their eventual capacity to handle with the same degree of efficiency 
loads up to four times greater than the work loads they could barely handle 
at the outset. This increase in efficiency of the team operation was undoubtedly 
due in part to increased motor effectiveness of individual crew members; how- 
ever, the most significant increases were due to the development of new 
Work procedures. New procedures in their turn introduced new relationships 
Of man to man and man to equipment, as well as new conceptions of the 
tasks to be performed by men and equipment. 

As the age of computers comes upon us, it becomes increasingly clear 
that machine procedures (in this case, computer programs) constitute a vital 
aspect of team operations. When man first picked up a club to defend himself, 
it was an extension of his arm and a hardening of his fist. The modern high- 
speed computer is an infinitely more complex extension of the human organism 
(compare Chapter 3). Already it is an extension of human memory and 
computational ability; soon it may be an extension of man's ability to con- 
Ceptualize or to recognize complex patterns. With much of memory, com- 
Putational procedures, thought, and perception delegated to machines and 
their procedures, the role of human beings in team efforts is substantially 
altered. But the greater the degree of delegation of these activities to machines 
the more obvious it becomes that machines and machine procedures must be 
thought of as team members, 
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These examples help convince us that the word *team" should be used 
to encompass human beings, work procedures, machines, and machine proce- 
dures as they interact in contributing to, or detracting from, the accomplish- 
ment of the defined goals of a System or subsystem. 


The Meaning of Team Functions 
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members. This identification of purposes being served, as opposed to a descrip- 
tion of the system in terms of actions or procedures per se, is important for 
several reasons. First and foremost among these reasons is that in almost all 
(if not all) systems the same purposes could be served by a great variety of 
different team organizations and work procedures. Because this is true, a 
failure to perceive a function or a malfunction may lead the system designer 
to miss an opportunity to improve team performance. To the extent to which 
the system designer foresees the functions which the team is to fulfill, he 
increases the choices of team organization which could accomplish them. Of 
vital importance, too, is the fact that system performance cannot be adequately 
evaluated if its functions are not understood. Last, but not least, is the fact 
that unless the important team functions are understood clearly, the training 
of team members is apt to be unduly unrestricted, as we shall point out 
subsequently. 

So much then for the meaning and significance of the term team func- 
tions. Let us now consider a definition of team training. 


The Meaning of Team Training 


lt is possible to define team training, in the broadest sense, as any 
experience in which a team engages which results in a change of team function, 
team organization, or team performance. This definition is made possible 
because of the following factors: (1) man-machine systems or teams are 
contrived to reach certain goals such as profit making, defense of the nation, 
Or the education of students; (2) our modern, complex man-machine teams 
encompass men, work procedures, machines, and machine procedures; and 
(3) team functions can be fulfilled by a variety of combinations of men, 
machines, work procedures, and machine procedures. Effective training will 
result in more efficient attainment of team goals. Put in other words, the 
evidence that a team has learned is in the changes or adaptations it makes as a 
result of experience. The experience may be a planned one, as in a training 
Program; or it may occur in the course of team operation—that is, the team 
can learn from experience. Furthermore, teams, like individuals, can have 
some bad experiences and can learn some bad habits. The learning which 
Occurs is manifest in changes in work procedures, changes in equipment, 
changes in machine procedures, and changes in operator proficiency. 

It is important to note here that this definition of team training goes 
Substantially beyond the current, generally accepted, concepts of team training. 
It does not limit itself to saying, “These are the machines in this system; these 
are the machine procedures in this system; these are the work procedures in 
this system; and the job is to teach the personnel about the machines, the 
machine procedures, and the work procedures, so they can make the system 
Work as it is supposed to work." Perhaps the most serious shortcoming in 
traditional team training efforts is the acceptance of work procedures, machine 
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procedures and machines as given. When attention is focused on functions, 
changes in machines, machine procedures, and work procedures assume a 
significance which is directly comparable with changes in operator pronciencya 

An alternative approach to team training which stipulates that the work 
procedures are fixed (as well as the machines and machine procedures) leads 
to a definition of the training problem solely in terms of increasing crew pro- 
ficiency in the fixed procedures. Such an approach may be referred to as com- 


ponent training because it focuses on the component (human, in this casc) 
and disregards the interaction between components. 


Improvement by Simple Machine Alteration 


Let us consider here some ways in which teams acquire new and more 
effective interactive routines by si 


cient team operation, and the kind of interaction adaptation which was brought 
about in each case. 
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amount of energy spent on different tracks increases, so does the difficulty 
of remembering just what has been done with any one track. Opportunity 
for error arises. 

A simple machine introduced to aid the lateral telles memory consisted 
of a simple 14"X16" column-and-row display board. The operator recorded in 
the extreme left column the designators of the tracks to be told, and in time- 
labeled columns to the right he simply made check marks to show when he 
had told a track. This display, in effect, "remembered" for the operator the 
actions he had taken on every track. He needed only to glance at any row to 
tell how much or how little telling of the track he had done and when he had 
last taken action. 

An unexpected benefit to the team operation arose from the fact that not 
only did the display let the operator know how he himself was distributing his 
energies but it also let the operator's supervisor know how the operator was 
distributing his energies. Before the development of the display board, the 
supervisor could sense what the operator was doing only by monitoring the 
telephone line for a period of time or by studying a log book kept by the 
lateral teller. The display board made the lateral teller's functioning highly 
visible and thus enabled the supervisor to give timely instruction. 

Consider next the rather profound system changes that transpired when 
Whyte (1948) introduced to the short-order restaurant industry the device 
known as the spindle or wheel, on which waitresses could place their written 
order checks. As Porter (1961) points out, the wheel first of all acts as a 
memory for the cook; he does not have to remember orders, the wheel does 
it for him. The wheel also acts as a buffer drum. The input rate and output 
rate need no longer be one-to-one. Ten waitresses may come to the wheel 
almost simultaneously, but the cook takes the orders one by one. The wheel 
also acts as a double queuing device. Waitresses no longer need to stand in 
line to put in their orders; the wheel stands in line for them. Moreover, the 
Wheel does not get the orders mixed up, but keeps them in proper queue 
sequence. Finally, the wheel also serves as a display of all the information in 
the system at a given time. The cooks, by having random access to the infor- 
mation, are enabled to organize their work around larger work units, such as 
the simultaneous preparation of three or four similar orders. The fact that the 
Order is recorded on a check, equally available to waitress and cook to check 
back upon when an error has been made, permits feedback and the consequent 


elimination of habitual errors. 


Improvement in Work Procedures 


Work procedures governing the interaction among team members may 
also undergo distinct improvement in effectiveness as a result of training within 
a system context. Let us consider here two,examples of this type of event. 

In a manual radar station, the scope reader "reads" the blips on the scope 
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to the plotter. The scope reader, by telephone, announces to the plotter the 
azimuth, range, and speed of a track. Traditional individual training might be 
directed at having the Scope reader attain greater accuracy and speed—that is, 
Or scores. Zero time and error scores, 
desirable. Under other circumstances 
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had gone off the air. Coming at him from inland was a second unknown, 
clearly in his own sector. Despite the fact that an inbound, unknown aircraft 
is substantially the more threatening, the senior director scrambled after the 
aircraft in his own sector. 

As it developed, the aircraft coming from inland was a friendly aircraft, 
which had been erroneously reported as unknown. The aircraft coming in over 
water was a hostile aircraft that completed its bomb run. When the operations 
officer at division headquarters telephoned to the site to inquire why a friendly 
aircraft was scrambled on and a hostile aircraft left unmolested, the answer 
was, “I’m responsible for what happens in my sector. If we're able, we help 
out Dixie [code name for the disabled sector], but I had only one interceptor 
left and I had to look after my sector first." Such an attitude exhibits an 
admirable loyalty, but fails to accomplish the task of air defense. As a result 
of it, the senior director had a one-trial learning experience. He learned that 
his job was a matter of defending against threatening aircraft, and not simply 
of defending his sector. 

So much then for the meaning of team training. A team consists of men, 
hardware, and procedures. All are subject to adaptation to the task as a result 


of experience. 


FUNCTIONS IN ESTABLISHED AND 
EMERGENT TEAM SITUATIONS 


[a] 


In considering the variety of system operations in which teams may 
engage, it appears desirable to distinguish a continuum of situations in which 
action can occur. One pole of this continuum consists of established situations; 
the other pole consists of emergent situations. An established situation is one 
in which (1) all action-relevant environmental conditions are specifiable and 
predictable, (2) all action-relevant states of the system are specifiable and 
predictable, and (3) available research technology or records are adequate to 
provide statements about the probable consequences of alternative actions. 
An emergent situation is one in which (1) all action-relevant environmental 
conditions have not been specified, (2) the state of the system does not 
correspond to relied-upon predictions, (3) analytic solutions are not available, 
given the current state of analytic technology (Boguslaw, 1961). 

It is clear that a specific team may be called upon to deal with situations 
Which vary along the entire length of this continuum. Team functions for 
established situations are tentatively planned for in the design of a system and 
are predictable at specific probability levels for varying states of system 
Operation and environmental conditions. They include (1) the jobs assigned to 
the men and machines of the system, and (2) the agreed upon procedures for 
getting these jobs done. Team functions for emergent situations are frequently 
ignored on the formal level, although usually anticipated informally by experi- 
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enced personnel. They include the actions which must be taken when com- 
munication channels unexpectedly fail, the procedures which must be followed 
to deal with an unanticipated input or threat or an operational crisis—for none 
of which are there “standard” or “book” answers, but which must be dealt 
with on the basis of adaptation to an immediate situation. 


Predicting Functions for Teams 
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problems of industrial productivity (Mayo, 1933, 1947a, 1947b). Related to 
these concerns are those regarding the flow of information in a system. To 
whom, at what rate, and in what order information should flow will loom as 
central issues in this viewpoint. 


The System-Model or System-Engineering Perspective. The advent of 
Operations research has formalized a somewhat different perspective. The 
fulfillment of any function has its cost in time, moncy, people, hardware, and 
the like. Operations researchers will want to examine the system to help 
achieve minimax solutions: minimum costs for maximum gains (Churchman, 


Ackoff, and Arnoff, 1957). 


Several Perspectives. Other writers might well add to this list of per- 
Spectives, or even categorize them differently. The major point to be made, 
however, is that the experts in system design all too often bring to their work 
à singleness of perspective, while what is needed is to bring to bear many 
perspectives on the problems of system design from the very outset. 

This involvement of many perspectives is probably most important in the 
design of today's large-scale systems. Consider, for illustrative purposes, the 
problems which face modern military systems. The advances in weaponry are 
So rapid and so bizarre that one cannot be certain of what a command and 
control system currently being designed will ultimately have to deal with. Add- 
ing to the difficulty, of course, is the fact that if our weaponry and control 
Systems are "good enough," presumably they will never have to be used, since 
they will provide “deterrence.” And if they are never used, no one will learn 
from experience whether or not the systems were effective. But in case they 
should have to be used, it seems wise to make certain that they are usable. 

Consider again the short-order restaurant. The physiological perspective 
would contribute to better system design by looking at such factors as the 
perceptual distinctness of printing on the menu, the symbology used on the 
checks, the lighting, the air conditioning, the physical layout, and the flow of 
work, The bureaucratic perspective would consider problems of delegation of 
responsibility, working hours, wages, benefits, employment policies, and so 
on. The interpersonal-relations perspective would consider the impact of 
employment policies on employees and on supervisory relations, the handling 
of customers during rush hours, the distribution of work between old hands 
and new employees, and a thousand and one other status problems. The 
system-engineering perspective might well be concerned with the inventory 
Problems involved, and seek a minimax solution which would involve the 
Optimum amount of money tied up in supplies and equipment. It might also 
Concern itself with the system design under conditions of overload. It would 
ask how the design would handle queuing, error, and omission problems. It 
Would trace the “information” from its printed form on the menu through its 
verbal form as spoken by the customer, its written form as scribbled by the 
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waitress, its display form on the spindle, its form as food, its form as money 
in the till, and so on through its total process. No doubt there are other 
perspectives in which team functions can be viewed. The more complicated 
the system, the more its design would seem to require a variety of perspectives. 


Emergent Functions 
In complex systems, characteristically, not all necessary situations 
à priori analysis. Every System will need methods 


tuations. There are five major methods available 
to organizations for dealing with these Situations: (1) the selection and use of 


managerial personnel, (2) the selection and use of equipment and facilities, 


(3) the formulation of Policy guides, (4) the improvement of system analysis 
and computer technologies, and (5) team training. 
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engineers determine the theoretical capacity of the equipment and then 
deliberately add a 50 percent increase in input-output capacity, because 
experience has taught them that human operators will sooner or later devise 
ways to make the equipment exceed its theoretical capacity. 


Formulation of Policy Guides. When a class of problems has been 
experienced over time by an organization, it becomes possible to formulate 
Policy guides. A policy guide describes a type of problem, specifies the values 
and interest to be protected, and relates these values and interests to the 
organization's basic goals. The ostensible purpose of a policy guide is to aid 
people in dealing with all occurrences. In practice, it characteristically attempts 
to provide an action rule for each event. Thus, what is intended as a policy 
guide often becomes in fact a set of work procedures. Since, however, many 
events have not in fact been anticipated or predicted, many critical situations 
Which actually do occur have no rule specified. It is for these emergent situa- 
tions that a policy rather than a procedure is required. 


Technology of System Analysis. A fourth method of dealing with 
emergent functions is through improvements in the technology of system 
analysis and in the technology of computer use. Christie (1956) points out 
that recent advances in these technologies are literally crying out to be used. 
To illustrate this, he notes that outmoded concepts of analyzing systems’ 
needs lead to the building of more and more highways as practically the sole 
answer to the problem of traffic congestion, especially in urban areas. Updated 
System analysis techniques would uncover the dynamics of traffic flow. Com- 
Puter technology could readily apprehend these dynamic changes and alter 
traffic-control signals to correspond wtih load and rate changes when and 
Where they occur. It is the uncovering of relevant information and its timely 
handling that constitutes the fourth avenue for coping with emergent functions. 
Improvements in these technologies are being incorporated extensively in the 
design of modern military command and control systems. 


Team Training. A fifth method of dealing with emergent functions is 
through team training. This we shall discuss in the following section. 


TEAM TRAINING TECHNOLOGIES 


[s] 


When one deals with established situations, many training techniques are 
available. There are all the conventional individual and group instructional 
Procedures, such as classroom lectures and audio-visual methods. A rapid 
exploitation of teaching machines and simulators may be expected in the 
immediate future. On the other hand, severesproblems are encountered when 
àn attempt is made to adapt customary training procedures to the needs of 
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team training as defined here. The tendency is strong in our culture to feel 
that if every man does his job, all will go well, and therefore all a man needs 
to know is how to do his job and to leave the next man’s job to the next man. 
But training for effective team performance, even in established situations, 


should incorporate a number of additional considerations, which we shall 
attempt to describe here. 


Orientation to Team Goals 


While they may be expected to vary a great deal from one organization to 
the next, and from one level of job to another, the team’s goals should be 
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changed. Errors decreased markedly, and the error rate stayed at a very low 
level. Understanding the consequences of error produced a significant change 
in the perspective of each of the cashiers. No longer were they simply dis- 
bursing funds; they were preventing expensive legal bills, preventing an 
indigent from public embarrassment, and preventing the agency from getting 
a bad name. 


Training in Interdependencies 


In organizations where the work flow is broken down into units of work 
(such as scope reading, plotting, movement identification, and so on) there 
will be interdependent relations between the team members. These inter- 
dependencies are seldom obvious from a chart of the work flow. As a conse- 
quence, they are seldom incorporated into training curricula; yet they are 
often the source of serious team errors. An example of such interdependence 
was described previously in the case of the scope reader who read plots to the 
plotter at too rapid a rate, resulting in a seriously inaccurate display of 
vital information. 

In one air defense crew it was discovered that the senior director was 
placing great emphasis on the rapid identification of tracks. The movements- 
identification operator was not only very competent but highly motivated. The 
team, however, was making a high proportion of scrambles on friendly air- 
craft. It seemed that the movements-identification operator was not being very 
efficient; observation of his behavior, however, indicated a high degree of 
efficiency. He was exceptionally fast at his work—so fast, it was finally dis- 
covered, that whenever the plotter made one little misplot, an interceptor 
Went up. This happened because the movements-identification man immedi- 
ately reported a track to be uncorrelated with a flight plan, thereby virtually 
requiring the senior director to declare it unknown and scramble an interceptor 
in short order. The movements-identification operator learned that his effective 
performance was indeed a matter of working with the plotter to check for 
possible misplots, rather than simply accepting the information displayed. 

Such examples emphasize the importance of the idea that in training 
Personnel for operations, the facts of interdependency should be incorporated 
into the curriculum and given appropriate stress. 


Training for Error Analysis 


One of the most difficult skills to train, yet one of the most important 
skills in dealing with team functions, is skill in the analysis of one’s own errors, 
For most of us, training in the notion that errors are bad and should be 
avoided began at home, was carried on in school, and continued in our worka- 
day life. Most of us, in order to avoid the negative consequences of an error, 
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adherence to standard operating procedures but which call for the application 
of a policy guide. Problem-solving discussions (rather than didactic presen- 
tations) might well help trainees to gain a problem-solving attitude. 


Training for Sensing Overload 


It is not infrequent in a team operation for one or more operators to be 
overloaded. When an operator is overloaded, errors of commission, errors of 
omission, and time delays are bound to occur. It becomes important that the 
operator learn to put aside the tendency to be a hero by trying to handle the 
entire job. He must learn w/ien he is reaching the point of being overloaded 
and when to call for help. At the same time, other operators must learn to 
sense when a team member is nearing an overload condition and be prepared 
to watch closely for situations in which errors occur and for which they can 
take compensatory action. 

During an air-defense system training exercise at one radar direction 
center, the team was given a very heavy load problem. After a period of time, 
the scope readers and plotters were "all fouled up." For the period of several 
minutes that were required for the surveillance officer to untangle the con- 
fusion, the senior director and the movements-identification operator took over 
the surveillance officers normal duties in addition to their own. They were 
unable to do so for long, but they could manage it for just long enough. They 
had sensed the surveillance officer's state of overload and responded pro- 
ductively to it. A characteristic response in a conventionally trained team 
might have consisted of an attitude colloquially expressed in the statement, 
"Tough— but it wasn't our fault.” 


Training in Adjustment Mechanisms 


When a team is overloaded, it may make adjustments of various types. 
The team may so operate as to permit queuing; that is, it lets things wait, or 
stack up, for a time. Restaurants have waiting lines; long-distance operators 
quote time delays; airplanes wait in line to take off. 

The team may engage in omissions; that is, it may not accept all the usual 
inputs to the system. A factory accepts no new orders; a restaurant accepts no 
one who does not have a reservation; the telephone lines give a busy signal. 

The team may so operate as to commit errors; that is, it permits certain 
errors to take place. The mail-order house finds it faster not to double-check 
all orders; the waitress brings relish, a tomato slice, and lettuce with your 
hamburger, whether it has been ordered or not; the telephone company per- 
Mits people to dial wrong numbers rather than jam up the service by routing 
all calls through operators. 

Another adjustment which may be adopted is filtering; that is, the team 
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particular times. The reasons are twofold. On the one hand it would be next 
to impossible to anticipate and incorporate into a curriculum the multiple 
uniquenesses to be found at the many individual job settings, even though the 
importance of training for these uniquenesses is openly acknowledged. Tradi- 
tionally, this particularized training is left for on-the-job training (OJT). 
Secondly, the time lags are great between changes in the environment of a 
system, the system's adjustment to the changes, and the incorporation of the 
changes into a school curriculum. To expect curriculum planners to foresee 
the changes in the system's environment and to foresee the changes which the 
system would make in adjustment to the changes would be to expect too much. 
For these reasons, then, the conversion of an individual operator into an 
effective team member is inherently the final responsibility of system operation. 

Much can be done to heighten operator and team effectiveness by well- 
conducted OJT. It should be noted, however, that OJT is characteristically 
designed and conducted to train operators in functions which are clearly 
established. OJT in narrowly defined individual tasks is not calculated to train 
Operators or teams to become effective agents for sensing the problems created 
by changes in a system's environment. It is normally not addressed to the 
problem of training operators or teams to become effective agents in changing 
work procedures, equipment, and equipment procedures, to meet the problems 
Created by environmental changes. For an understanding of a training tech- 
nology which does train for continuing adaptation to emerging functions, we 
must turn to the system training technology as described in Chapter 13, or as 
otherwise described by Biel (1958) and Goodwin ( 1957). 

At the time this chapter is being written, system training is being con- 
ducted solely within military organizations. These organizations are complex, 
and they use equipment and procedures quite unfamiliar to most readers. 
Since it is often difficult to understand the training technology when it is 
described in terms of unfamiliar activities and procedures, the discussion of 
System training technology which follows is embedded in a system with which 


almost every reader has had experience. 


E] DESIGNING SYSTEM TRAINING 


The system chosen as the vehicle for illustration of how system training 
technology is applied and how it operates is one that occurs in a public 
elementary school. The training need chosen for illustration is the need of the 
team members—pupils, teachers, administrators and maintenance personnel— 
to meet such emergencies as a fire or an explosion in the school building. As 
the reader proceeds, a helpful exercise would be to generalize from this discus- 
Sion how the technology might be applied to other systems such as hospitals, 
airlines, telephone companies, railroads, or department stores, or to any other 
Systems with which he has had enough experience to understand the structure, 
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Let us presume for the moment that a given school has been found to be 
vulnerable, among other things, to fire in a closet under a wooden stairwell. 
Consultations with fire experts reveal that the following sequence of events 
might be expected were a fire to start in the closet at ten o'clock in the 
morning:? 


10:00 Spontaneous combustion takes place. 

10:08 Smoke appears under closet door and at edges of the door. 

10:15 Last possible moment that closet door could be opened by someone 
in erect posture without injury by flames. 

10:16 Wisps of smoke begin to seep from under stair steps. 

10:20 Flames appear at edge of closet door. 

10:23 Flames have broken through closet ceiling and are beginning to burn 
stairway steps. 

10:24 Wire to fire-alarm system destroyed; fire alarm impossible. 

10:26 Heavy smoke fills stairwell and second-floor corridors. 

10:30 Heavy smoke fills all second- and third-floor corridors. 

10:32 Stairwell in open flames but still passable. 

10:36  Stairwell no longer passable. 

10:38 Flames race across second floor. 

10:43 All stairwells to second and third floors cut off by impassable flames 
and lethal heat; escape possible only through exterior windows. 


What would happen in a real-life situation, of course, would depend on 
Such factors as the time when the fire was discovered, the time it was reported 
to the principal's office, the time the fire department was called, the time the 
fire alarm was sounded, and the appropriateness of any actions to the condi- 
tions at the moment. An exit might be available to first-floor classes but be 
blocked to second-floor and third-floor classes. Alternative routes might be 
necessary, but the people who would need to select them might have no way 
of knowing whether they were opened or closed. 

Clearly, the training need is far beyond that of learning to march in 
orderly columns of twos to a designated space. In real disasters, some children 
Will panic and, upon reaching the safety of the outdoors, will run for home. 
Parents and curiosity seekers will crowd around the school. Police will form 
in cordons. Hours and perhaps days may be needed to restore order. 

Let us now consider how a suitable training problem might be designed, 
produced and run for such a public school disaster. 


Functions in Which Training Is Needed. First, one would ask: for what 
team functions is training needed and what priorities does each have? Let us 
suppose that this has been discussed, and that these functions have been 
settled upon with the priorities listed: 


? The events listed are deliberately distorted here for purposes of illustration. For an 
accurate description of how fires develop, the reader is referred to Operation Scho. l 
Burning, National Fire Protection Association, 60 Batterymarch Street, Boston 10 Ma s 


1959. This book reports on the results of test fires in a three-story school building ass. 
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needs which are foremost will be an additional factor to be considered in the 
design of a specific problem. 

It is quite possible to imagine a problem in which a simulated disaster 
has already occurred and all the children are out in the street or yard. The 
training task might well be practice in getting out of the way of oncoming fire 
engines, of finding distraught parents, and of providing aid to the injured. 

As the training needs of the school children change, the training needs 
of the community evolve. If fire is simulated, fire-department personnel may 
Wish to participate in the exercise. If fire-department personnel start rolling fire 
trucks up to a school, crowds will gather. The police department may wish to 
use the opportunity for the purpose of permitting their own personnel to get 
experience with handling crowds or to pass out pamphlets telling onlookers 
how they hamper emergency efforts. Red Cross units may wish to simulate 
the setting up of emergency units. Hospitals may wish to participate for the 
Purpose of testing their disaster plans. 


The Problem of Cost. The costs involved in such exercises can vary 
Over a wide range. The direct costs of some simulated events could be pre- 
dicted with great accuracy. Less predictable would be costs arising indirectly 
from the degraded emergency posture of the fire department and the police 
department. There would be costs in the loss of educational time. In the final 
analysis, well-run training exercises would certainly exceed the cost of simply 


marching children out of a building. 


The Training Team. When a problem has been designed and is ready 
to be run, the training team must be prepared. This is the unit which will 
contain observers and umpires. All results which are critical in terms of the 
functions being trained must be recorded. For example, in the fire-in-the-closet 
problem an observer-umpire stationed near the closet would observe and 
record the time and action of anyone who does or could take any action with 
regard to the smoke coming from the closet. 

To record appropriate events, observer-umpires would need to know the 
problem design in advance and to have discussed thoroughly the consequences 
of the probable actions that might occur at their observation points. Addi- 
tionally, they would need forms on which to record their observations quickly. 
The recording of observations should not be allowed to interfere with observ- 


ing or umpiring. 


Announcing the Exercise. The presence in the school building of the 
Observer-umpires would very soon come to be a tip-off to the children that an 
exercise was imminent. This is a matter of no concern, since the children 
Should be informed that an exercise will be held, but should not be told when 
Or what type of emergency will be simulated. ^ 

There are two principal reasons for informing the children that an exer- 
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cise will be held. First, it is desirable that the children be aware of the actions 
they are taking and that they think about their actions as they perform them. 
The training is aimed at developing a thoughtful and informed response rather 
than an uninformed, emotional response to emergencies. The second reason 
for announcing the exercise is to avoid panic. Panic could possibly ensue in the 


face of realistic simulation, were children not prepared for it either by knowl- 
edge of the exercise or by training. 


Postexercise Discussion. The conclusion of the exercise is the time to 


provide knowledge of results to the trainees, and in this operation a number 


of factors are of fundamental importance. At any one critical point in some 
problems, a number of different responses m 


of the purposes which must be served. If the situation develops somewhat 
differently, some of the responses may not 


be adequate while others, entirely 
new or previously inadequate, may now be adequate. In other words, there 
may be a number of correct responses at 


ay be entirely adequate in terms 


leaders should avoid using psycholo nd to produce 
habitual responses. Rather, the discussi 


On the other hand, where 


an atomic explosion. This pres ly is the best response they can make 
under such circumstances. 

Another problem for the trainers is to be found in the mechanics of 
setting up the discussion groups. Should children be h 
the teachers? Should all persons assemble in one location? Should the break- 
down of groups be by home rooms? 


—Robert Boguslaw and Elias H. Porter - All 


a detailed discussion. If the problem unfolds in such a way, for example, that 
the chief difficulty is in communication among the janitorial and teaching 
staffs, it seems pointless to involve the children in postexercise discussion. On 
the other hand, if the principal difficulty arose because the children were noisy 
and orders could not be heard, one might judge it best to deal with the children 
in small, intimate groups. Or, should the action taken by a single child turn 
out to be the most crucial event in the problem, possibly a short assembly 
presentation might be judged most effective and economical. 

One final point relates to postexercise discussions. Chapman (1960) 
points out that in the RAND Corporation System Research Laboratory 
studies, marked improvements in crew performance at times followed post- 
exercise discussions in which no apparent problem-solving behavior took 
place. These sessions seemed to follow problems which the crew believed they 
had handled poorly. The postexercise discussion periods were characterized 
by complaints and gloom. Porter (1950) notes a similar phenomenon in 
Psychotherapeutic interviews. The implication is that the learning process pos- 
Sesses at time a strong emotional component. Porter (1961) attributes this 
type of behavior to the process of learning how to learn, and points out that 
the success of a postexercise discussion cannot be judged solely on the intel- 
lectual problem solving which takes place during the period. A second impli- 
cation arises for the discussion leaders; they should be prepared to handle the 
emotional content of a discussion as well as the intellectual. 


E A RESEARCH PERSPECTIVE 


A recent commentary on small-group research, made by a distinguished 
researcher (Borgatta, 1960, p. 180), notes the enormous amount of "waste 
type" publication which seems to occur in this field. He points out that thou- 
sands of articles have been published on sociometric tests, projective tests, 
Social perception and learning. The implication is clear that while much has 
been said, most publications on these topics have contributed little of practical 
Usefulness to system designers. : 

Others have observed that the vast majority of studies directed toward 
increasing the efficiency of team performance and developing à greater under- 
standing of the way small groups function have not been particularly relevant 
for improving the performance of typical working teams (Glanzer and Glaser, 
1961; Klaus and Glaser, 1960). 

Criticism of research as "not significant" seems to be most cogent when an 
emergent situation is studied as if it were an established situation. Conversely, 
Criticism of theoretical investigations as "not true" is most cogent when an 
established situation is studied as if it were emergent (Boguslaw, 1961). 

It is clear that research programs must avoid errors of both types if 
Progress is to be made in improving our ability to increase the effectiveness 
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of teams. Frequently this is done through research specialization in studies of 
established situations, on the one hand, or emergent situations, on the other. 
For example, a program of research at the Team Training Laboratory of the 
American Institute for Research seems to be concerned primarily with estab- 
lished situations. A report describing this program finds it convenient to 
differentiate between the terms team and small group. 

According to the definition used in this program, teams (which represent 
the focal interest) generally: 


1. are relatively rigid in structure, organization, and communication net- 
works; 


2. have well-defined positions or member assignments so that participation 
in a given task by each individual can be anticipated to a given extent; 

3. depend on the cooperative or coordinated participation of several spe- 
cialized individuals whose activities contain little overlap and who must 

each perform their task at least at some level of minimum proficiency; 

are often involved with equipment or tasks requiring perceptual-motor 

activities; 

. can be given specific guidance on job performance based on a task 

analysis of the team's equipment, mission or situation. 


Small groups, on the other hand, in terms of the definition used in this 
program, generally: 


1. have an indefinite or loose structure, organization, and communication 
network; 
2. 


have assumed rather than designated positions or assignments so that 
each individual's contribution to the task is largely dependent on his 
own personal characteristics: 


depend mainly on the quality of independent individual contribu- 
tions and can frequently function well even when one or several 
members are not contributing at all: 


4. are often involved with complex decision making activities; 
5. cannot be given much specific guidance beforehand since the quality 


and quantity of participation by individual members is not known." 
(Klaus and Glaser, 1960, p. 2-3.) 


Where it is assumed or judged that sudden changes or disruptions will not 
be a critical part of the task with which a team must cope, but rather that the 
team's own environment, operational state, and procedures will be essentially 
predictable, certain advantages accrue to the researchers. In the first place. 
they can abstract from real-life team tasks those central missions which 
characterize the operation and then represent them in simplified form within 
the laboratory. To abstract and simplify operations often saves a great deal of 
money, especially where complicated, expensive machines are involved in the 
real-life situation. Simplifying the operation means that “crews” can be experi- 
mented upon in finite time periods. Additionally, simplified operations can be 
kept track of more readily by the researcher. He does not, for example, have 

L3 


? Reprinted by permission of the authors and the American Institute for Research. 


—Robert Boguslaw and Elias H. Porter - 413 


to worry about nonprogrammed personnel changes due to terminations, trans- 
fers, sick leaves, and the like. He can more carefully control the events which 
would influence team performance beyond the experimental variables. 

There are, on the other hand, some obvious disadvantages to an approach 
Which deals solely with the established functions of teams. A central dis- 
advantage arises from the loss of possibly significant variables which accom- 
Panics the simplified representation. As systems increase in complexity, it 
often seems that they become vulnerable to frequent breakdowns and self- 
generated problems, for which compensatory action must be taken. Excessive 
abstraction and simplification may result in a loss of opportunity to come to 
grips with problems which exist in the real world—that is, system problems 
which require solutions—if effective team training is to be achieved. Where 
total systems are subject to stress by overload or by sudden degradations, 
Tepresentations of established functions do little to help the researchers 
develop concepts for training teams to meet these overloads and stresses. 

An example of a research program which seems specifically to have been 
concerned with improving team performance in situations of the kind we have 
described as emergent is that of the RAND Corporation System Research 
Laboratory's air defense experiments (Chapman et al., 1959). These experi- 
Ments postulated sudden changes, disruptions, unpredicted and unspecified 
interactions between subsystems, and rapid adjustment to these stresses, as a 
Critical part of the team's ongoing operation. The experimenters began with 
the premise that “exploiting the ability of men in organizations to learn 
requires . . . understanding organizational adaptation" (Chapman et al., 
1959), They were dissatisfied with exploration of ad hoc hypotheses and 
addressed themselves to the problem of looking at the system as a whole to 
find Significant variables rather than examining variables separately and post- 
Poning their integration. They described their experiments as a search for a 
framework for comprehending organizational behavior rather than as tests of 
Particular hypotheses. 

Engagement in studies of this kind involved extensive simulation of a 
System’s continuously varying environment and internal structure. It is reported 
that the objective of getting organizational behavior into the laboratory was 
realized, 


Each of the four crews gradually came to behave as if it were in a real 
life situation. Crew members became deeply involved with the organiza- 
tion's goal and its successes and failures. During enemy attacks, the noise 
level in the station rose, men came to their feet, and the excitement was 
Obvious. Crew members reported restless nights and bad dreams— 
attackers boring in without an interceptor available. On one Occasion, an 
Officer slipped while stepping off a dais and broke his leg. We were not 
aware of this event for some ten minutes because there was no perturba- 
lion in the crew's activity during the attack in progress. He was back the 
Next day, cast and all, because as he said, they couldn't get along without 
him. The member of each crew became an integral unit in which many 
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interdependencies and coordinating skills developed. And each crew 
learned to perform more effectively. This learning showed itself in pro- 


cedural shortcuts, reassignment of functions, and increased motor skill to 


do the job faster and more accurately.* (Chapman et al., 1959, pp- 
263-264). 


This richness of realism permitted the problems of subsystem interaction 
to be studied in meticulous detail (see Figure 11.1). Additionally, it forced 


the researchers to formulate concepts at higher levels of generality such as 
Chapman's (1960) concepts of a processing coercion, a system inertia, aN 
effectiveness coercion, and an adjustment c 


ycle; concepts which view the team 


vistas in team training. 
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ming, in machine implementation, and in the conceptual structure necessary 
for communication among all elements of modern teams, are all part of the 
new look which must be given to the problem of team training in contem- 


porary large-scale systems. 
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THE HUMAN COMPONENTS OF SYSTEMS MUST 
be given the skills they need by the techniques of indi- 
vidual and team training, in order to have the capa- 
bilities required to perform their tasks. But some 
determination must also be made to test whether these 
procedures have been truly and thoroughly effective in 
establishing these capabilities—to find out what indi- 
viduals and teams can do in the system context. These questions are answered 


by techniques of assessing human performance called proficiency measure- 
ment, which are discussed in this chapter, 


Measurement of the proficienc 
aspect of system development, 
within the entire development-ut 
ment are used to evaluate the p 


y of human operators is an important 
Which has a number of different purposes 
ilization cycle. The results of such measure- 


Tocesses of selection and training, 
to estimate the contribution of human competence to total system 


ance. Measures of individual Proficiency also establish 
one can refer the gains in System effective 
experience and by team training. 

Some of the basic Concepts of proficiency measurement have been 
familiar to psychologists for a long 


weighting of component meas 


as well as 
perform- 
à base line to which 


sures of on-the-job 
d operations, have 


xamination of some of the 
en developed 


within modern systems, a ms these h 


and employed 
future technological devel 


nd some of the proble àve suggested for 
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PROFICIENCY MEASUREMENT: 
ASSESSING HUMAN 
PERFORMANCE 


Robert Glaser 
and David J. Klaus 


The output of the human component in a man-machine system requires 
s contribution to total system performance 
imary interest in the development of a 
System is in perfecting the performance of the system as a whole and not 
with isolated man or machine components, the proportion of variability in 
System capability attributable to the human component requires the careful 
assessment of this component's contribution to system performance. Use of 
total system performance measures to evaluate contributions of the human 
component frequently confounds the information obtained, since informa- 
tion available from knowledge of the performance of the total system provides 
only indirect assessment of any individual component. Nevertheless, measure- 
ment of individual component performance is necessary in the assembly and 
analysis of the total system. 

As an illustration of the meth 


measurement and assessment if it 
is to be effectively utilized. While pr 


ods used in the assessment of the human 
component in a total system, let us consider the performance of the driver 
in a man-machine system consisting of a driver and his automobile. Most 
of the measures that might be considered—such as speed, distance traveled, 
and ability to stop—are actually dependent upon interactions between 
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the human and the mechanical components in the System. Total system 
performance is not only dependent on the skill of the operator, but on the 
capabilities of the machine in the system as well. To the extent that mechani- 
cal components can differ, the results of any comparison will be attenuated 


with the same automobile, many additional 
S of the two drivers can be accu- 


s? uld it measure skill in heavy traffic, 
€ where the influence of nonpartici- 
the drivers be examined on relevant 


s which arise in assessing 
an-machine system. Ac the number 
» Increase and as the environment in 
controllable, accurate assessment of 
cult. This chapter considers 
d in Proficiency measurement 


tions. ures in several Practical situa- 


A : aracteristic. ies 
of present performance, with emphasis on the me Speo propertie 
In contrast, aptitude measures derive their mea 
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relationship between present performance and the future attainment of speci- 
fied skills. 

Measures of proficiency, then, sample desired terminal or criterion per- 
formance, such as the proficiency of a radar operator, an auto mechanic, or 
a student of French. Aptitude tests, on the other hand, need not necessarily 
involve the measurement of terminal behaviors, and any aspect of behavior 
can be used for aptitude measures so long as it can be hypothesized that it is 
predictive of future performance. Proficiency measures are restricted to what 
can be called criterion behavior, but often such behavior can be used suc- 
cessfully for predictive purposes as well as for assessment. An individual's 
attainment in high-school geometry, for example, may indicate how well he 
can be expected to do as a navigator or architect. While this use of proficiency 
measures for predictive purposes represents an important contribution in the 
development of an effective man-machine system, the topic is more properly 
considered as an aspect of the selection and classification techniques described 
in Chapter 7 of this book; accordingly, the main emphasis of this chapter is 
on the measurement of proficiency for assessment purposes. 


E THE CHARACTERISTICS OF PROFICIENCY MEASURES 


The behavior of the human component in a system can be evaluated or 
assessed for several purposes. In order to understand these uses of proficiency 
measurement, the kinds of information conveyed by a proficiency test must 
be examined. The scores from proficiency tests are used both to indicate 
(1) the degree to which an individual has attained criterion performance— 
for example, whether man A can satisfactorily complete a maintenance check 
On a particular type of radar; and (2) the relative ordering of individuals 
With respect to a given task—for example, whether man B assembles fuses 
more quickly than man C. The principal difference between these two uses 
lies in the standard used as a reference. Criterion-referenced measures depend 
On an absolute standard of quality while norm-referenced measures depend on 


à relative standard. 


Criterion-Referenced Measures 


Underlying the concept of proficiency measurement is a continuum of 


Skill ranging from no proficiency at all to perfect performance. In these terms, 
an individual's proficiency at a given task falls at some point on the con- 
tinuum, as measured by the behaviors he displays during testing. The degree 
to which his proficiency resembles desired performance at any specified level 
is assessed by criterion-referenced measures of proficiency. The standard 
against which an individual's performance is eompared, when measured in this 
manner, is the behaviors which define cach point along the underlying skill 
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continuum. When used in this way, the term “criterion” does not necessarily 
refer to final on-the-job behavior. Criterion levels can be established at any 
point in training where it is necessary to obtain information as to the et 
quacy of an individual's performance. Many jobs, for example, involve severa 
grades or levels of skill. A machinist can be categorized as an apprentice, n 
journeyman, or a master at his trade. The specific behaviors implied by eac 

of these levels of proficiency can be identified and used to describe the specific 


tasks an individual must be capable of performing before he achieves one of 
these skill levels. It is in this sense that me 


asures of proficiency can be 
criterion-referenced. 


continuum of attainment usually 
, in that a master machinist is also 
apprentice and journeyman levels. 


Proficiency. Measures which assess 
dard thus provide information as to 
h is independent of the performance 


à criterion stan 
the degree of competence attained whic 
of others. 


Norm-Referenced Measures 


can convey inform 
an individual compared with 


of other individuals as well as 
with respect to a standard of i 7 instances where an individual's 
relative standing along the un i i 
of measurement, reference ne to the criterion behavior. Edu- 
cational achievement examinations, for exam 
for the purpose of ordering individuals in à class or school, rather than for 
assessing their attainment of i 


of other members of the gro 
performance of the individual 


how proficient either of the 
distinction between criteri 


attainment is similar to those previously made by Flanagan (1951) and Ebel 
(1960) with respect to education: i 


— Robert Glaser and David J. Klaus - 423 


most achievement measures currently employed in education are norm- 
referenced, and thus do not provide the degree of information made available 
by the use of criterion-referenced measures. 

In education, norm-referenced measures of achievement are often 
referred to as "grading on the curve." Knowledge that a particular individual 
received an A grade in a course indicates that individual's relative standing 
or rank in the group, but does not indicate the degree to which he attained 
criterion performance. Perhaps the prevalence of this method of grading owes 
its existence to the difficulty encountered in attempting to specifically itemize 
the criterion behaviors being aimed for in the course of instruction. Both 
criterion-referenced measures and norm-referenced measures, however, indi- 
cate the desire to avoid assigning an arbitrary score to represent an individ- 
ualľ’s proficiency level. The utilization of course objectives as a standard 
against which to interpret a measure, as well as the use of group averages as 
the standard of performance, serve to objectify proficiency evaluation by pro- 
viding a meaningful reference for an obtained achievement test score. 

The expressions "percent" and "percentile" further illustrate the dif- 
ferences between criterion-referenced measures and norm-referenced meas- 
ures. When criterion standards are used, it is reported that Johnny knows 
80 percent of fifth-grade spelling words and Jimmy knows 70 percent; here, 
we have information both as to how they compared with one another as well 
as having information as to the content of their performance. When norm- 


referenced measures are used, it is reported that Jane was at the 90th per- 


centile of her fifth grade class in spelling while Joan was at the 40th per- 
centile. These scores convey information only as to the relative standing of 
the two students. It is apparent, then, that criterion-referenced measures are 
the more powerful of the two and must be used when information as to both 


behavioral content and rank in a group are to be obtained. 


Standards of Reference 


As is implied by the term “norm,” norm-referenced standards of pro- 
ficiency are based on data collected on the performance of a number of 
individuals having similar backgrounds and experience. Whether standard 
deviation scores, percentiles, or ranks are used to describe the position of a 
given individual depends largely on the kinds of data that are collected and 
the use to be made of the information. The size of sample used to establish 
the standard for norm-referenced scores is one of the factors which strongly 
influences the stability of the standard; too small a sample can yield measures 
of central tendency and variability that poorly approximate actual population 
values. In the same way, samples selected from among groups having widely 
differing backgrounds may fail to be representative of the population as a 
whole. Since the standard of performance, is established by the group and 
the standing of an individual is determined by his relationship to the group 
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ured by the number of items he turns out in a ten-minute period which fall 
within specified tolerance limits. Observing his output, one may determine 
that his production rate is 20 acceptable items per period. This raw measure 
obviously conveys little information as to his proficiency, except that he is 
turning out some product. In order to evaluate his performance, his score 
must be compared with some reference base. The rate of the worker across 
from him could be used, but such a restricted sample of a norm group may 
yield a comparison of limited value. An adequate sample of workers in similar 
jobs could provide a reasonably stable basis for measurement, however, and 
would permit the determination of any worker's relative standing. In many 
instances of proficiency assessment, norm-referenced measures are the only 
measures available. An individual's supervisory skills, for instance, are diffi- 
cult to evaluate in terms of specific criterion behaviors and without reference 
to the performance of other supervisors. 

Evaluating the proficiency of the assembly-line worker by means of 
criterion-referenced measures conveys information as to the individual's out- 
put when compared to specified performance standards as well as to his 
relative standing in a group of similar individuals. A criterion-referenced 
measure of the worker's proficiency can be based on a standard either of 
System requirements or of system capacity. Performance on the assembly line 
might be assessed in terms of the minimum human component output neces- 
Sary for system function, assuming that a later component in the assembly- 
line system requires the processing of at least some minimum number of units 
per time period (às, for example, an annealing oven, which depends upon a 
minimum number of items passing through it to maintain proper tempera- 
ture), In such a case, the assembly worker's output must equal the require- 
ment for the system to produce as planned. At the other extreme, the indi- 
vidual's proficiency could be compared with a standard representing system 
Capacity. If the system operated in such a way that the worker was supplied 
with a maximum of 40 items per time period, his output of 20 items for each 
period represents a proficiency level of 50 percent. Were the worker's rate 
to exceed the capacity of the system—that is, were he to have an output 
capability of 50 or 60 items per time period—the output of the system would 
not be increased and the additional proficiency would not represent any fur- 
ther contribution to total system performance. In certain systems, where the 
efficiency of mechanical components do not impose a limitation on system 
output, there is no upper limit to the potential contributions of the human 
component. There is probably no specifiable upper limit, for example, on 
such skills as inventiveness in systems where these behaviors contribute to 
over-all output. 

In applications of proficiency measurement to educational situations, 
Where it is generally the case that neither minimal requirements nor maximum 
effectiveness of the individual student are routinely specified, the selection 
of suitable standards is frequently difficult. Typically, school achievement is 
assessed by norm-referenced measures—that is, by "grading on a curve" of 
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between groups treated differently and to minimize the differences between 
individuals in any one group. Such a test will be sensitive to the differences 
produced by treatments. For example, a test designed to demonstrate the 
effects of training would be constructed so that it was uniformly difficult for 
those taking it before training and uniformly easy after training. The content 
of tests used to differentiate treatments should be maximally sensitive to the 
effects anticipated from the treatments in terms of performance changes. 

In essence, the distinction between proficiency tests used to maximize 
individual differences and one used to maximize treatment or group differ- 
ences is established during the selection of test items. In constructing a pro- 
ficiency test to differentiate among individuals at the end of training, for 
example, it would be possible to begin by obtaining data on a large sample 
of items relating to training objectives. When the data were analyzed, it 
would be found that some items were responded to correctly by only some 
of the individuals in the group, while other items were answered correctly 
by all members of the group. These latter items, which failed to differentiate 
among individuals, would be eliminated, since their only effect was to add a 
numerical constant to every score. Those items remaining would serve to 
discriminate among individuals and thus yield a distribution of scores that 
Was as large as possible considering the number and type of items used. If the 
test were constructed for the purpose of observing group instead of individual 
differences, the selection of items would follow a different course. In an 
instance where training was the treatment variable involved, for example, it 
would be desirable to retain items which were responded to correctly by all 
members of the post-training group but which were answered incorrectly by 
students who had not yet been trained. In a test constructed for the purpose 
of differentiating groups, items which indicated substantial variability within 
either the pre- or post-training group would be undesirable because of the 
likelihood that they would cloud the effects which might be attributable to the 
treatment variable. In brief, items most suitable for measuring individual 
differences in proficiency are those which will differentiate among individuals 
all exposed to the same training or other treatment variable, while items most 
suitable for distinguishing between groups are those which are most likely to 
indicate that a given amount of training or some particular treatment was 
effective. In either case, samples of test items are drawn from a population 
of items indicating the content of performance; the particular item samples 
that are drawn are those most useful for the purpose of the measurement 


(Hammock, 1960). 


Assessing Individual Differences 


The measurement of differences in individual performance is desirable 
at several stages in the development and utilization of the human component 
in a system. In a sense, the human component is “manufactured” by the proc- 
ess of training. As training progresses, the performance of individual trainees 
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system operation should not be overlooked. The test used should not neces- 
sarily have an arbitrary ceiling which precludes the evaluation of the indirect 
effects of training experiences, such as transfer of training and creativity. 


System Performance. Once the individual is on the job and functioning 
as a system component, periodic proficiency measurement is used to check 
performance adequacy and to assess whatever changes may have occurred 
as a result of field experience. Tests used for this purpose must have a wide 
range of coverage, since it should be possible to identify instances of enhance- 
ment in performance as well as instances of skill deterioration. 

The use of proficiency measures for any of these purposes assumes that 
the test user has knowledge of the intent and objectives of training. If the 
stated objectives of the learning experience are unclear, it is impossible to 
decide what behaviors ought to be measured. Frequently, it is found that the 
Objectives of training are either (1) defined in an overly general or ambig- 
uous way, or (2) expressed in terms which are difficult to measure. If the 
former is the case, it is probably equally true that the objectives of the system 
are ambiguously defined, and an effort must be made to define them rigor- 
Ously enough for performance measurement. If the latter is primarily the 
Case, the proficiency tester must devise a measurement situation in which the 
required behaviors are elicited for assessment. 


Assessing Group Differences 


The use of proficiency measures are not limited to the evaluation of the 
Present performance of an individual; they may also be used for evaluating 
the treatments or conditions which might be imposed to produced a given 
level of performance. These treatments can refer to any manipulation which 
may influence the proficiency of the human component, such as the training 
method, length of instruction, design of displays and controls, organization 
of the system, working conditions, and climate or environment. As noted 
above, when proficiency measures are used to distinguish between treatments 
by means of the differential performance of two or more groups, they should 
be maximally sensitive to the particular effects expected from the treatments 


being considered. 


Group Comparisons. Two or more treatments can be compared with- 
Out reference to a criterion standard in order to determine which of the 
treatments yields the highest level of proficiency. Two designs for dials can 
be contrasted, for example, by observing the proficiency of groups of individ- 
uals as they perform using the dials. The results of such a study can then 
be used to evaluate the relative effectiveness of the two designs and to decide 
Which of the dials might be most profitably) incorporated into the system. 
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Several methods of training can be compared in this fashion; the methods can 
then be ranked as to their relative efficiency. In the conduct of these com- 
parisons, no reference need be made to a criterion standard. Whether any of 
the treatments being investigated is adequate in terms of system require- 


ments, however, cannot be determined without employing criterion-referenced 
measures of man-machine proficiency. 


Group Attainment. 
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DEFINITION OF THE BEHAVIOR TO BE MEASURED 


Identifying the Performance to be Assessed 


: Y asurement, the task of a radio 
operator in receiving code, for example, can be analyzed in terms of speed 
accuracy, neatness of transcription, and other components. Specification of a 
similar set of defining behaviors for a Production supervisor in a factory is 
obviously more difficult, In order to 
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components of a particular job, and thereby accomplish the first step in the 
development of a proficiency test, it is essential to conduct some sort of task 
analysis yielding defined performance specifications which can be observed 
and measured. Techniques for carrying out the kind of systematic task analy- 
Ses basic for accomplishing proficiency measurement are described in 
Chapter 6. 


Purpose of Measurement. Once a task analysis is completed, a deter- 
mination must be made of the specific samples of performance to be selected 
for evaluation. The sampling procedure employed depends, to a large extent, 
upon the use to which the evaluation is to be put. If proficiency tests are to be 
used as predictive instruments—that is, to predict future performance on the 
basis of the present behavioral repertory, then the content of the test should 
sample those tasks which have been found empirically to discriminate among 
levels of proficiency in terms of future performance. For selection purposes, 
the test content need not measure all aspects of present performance, but only 
those which are demonstrably useful for enhancing the prediction. Those 
aspects of proficiency which are necessary for adequate job performance, but 
Which are performed reasonably well by all men tested, contribute little 


toward diflerentiating or predicting performance. 

On the other hand, consider a proficiency test used to assess present 
performance for placement purposes. The objective of measuring performance 
in this instance is to assess skill level so that an individual may be entered 
into a training program at the proper point or assigned to a task which he is 
capable of completing satisfactorily. The behavior sampled on such a test 
must incorporate not only those aspects which reflect differential levels of 
competence, but also those aspects of performance which comprise the basic 
Performance requirements necessary for all individuals to achieve the end 
products of training or assignment objectives. 

A different use of proficiency measures, the diagnosis of training diffi- 
Culties, requires still another definition of the behaviors to be sampled. In 
this case, it is necessary to obtain information about component behaviors 
which contribute to a later or final performance measure. The basis for sam- 
pling behavior for this purpose is to study the causes of inadequate perform- 
ance so that these causes can be reflected in the test situation. For such 
diagnostic purposes, the content of the test must not only involve performance 
Output, but must also reflect the component behaviors which contribute to 
the output. 

Performance measures require still a different emphasis when they are 
Used to assess the terminal behavior specified as the final training product. 
In this instance, the concern is less with diagnosis, and more with the deter- 
mination of performance adequacy with respect to standards of proficiency. 
For tests of terminal performance, the behavior sampled must be selected in 
terms of the stated objectives of the training program and thus reflect the 
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Quantification of Performance 


Measurement necessarily implies quantification. This means behavior 
which is to be assessed not only must be overt, but also must be amenable 
to having numbers assigned to it. The simplest instances, of course, are those 
in which the component behaviors which contribute to proficiency are either 
present or absent from the individual's or the system's performance. It is 
possible to count these events, such as the number of items produced per time 
period, the number of contacts a supervisor has with each subordinate, or the 
number of errors made during a battle exercise. In many other cases of pro- 
ficiency measurement, it is possible to quantify observations by employing 
Simple scale values, such as the time necessary to troubleshoot a radio 
receiver, or a marksman's score, where a given shot can be assigned a score 
of zero to ten depending on how far the shot was from the bull’s-eye. 

Measurement becomes more complex, however, when the important 
aspects of the behavior being assessed are qualitative rather than quantitative. 
For example, the skill of a pilot in reading an altimeter can be evaluated 
in terms of how many feet his readings are in error, but his skill in landing 
an aircraft cannot be effectively measured by a simple counting or measuring 
Procedure. In cases such as these, it is necessary to translate observable per- 
formance into some numerical scale. The two most popular methods for 
quantifying judgments are by the use of rating scales and check lists. 


Rating Scales. In applying rating-scale techniques to the evaluation of 
Complex activities, major components of the task are first identified. Each of 
these job segments is then defined in terms of a skill dimension ranging 
One or more judges are then asked to observe an indi- 
Vidual’s performance and select a point on the scale which is most representa- 
tive of the performance of the man being assessed. The most effective rating 
Scales are those which define the extremes of the behavior being observed 
and, if possible, provide descriptions of intermediate points along the dimen- 
sion. To a large extent, the adequacy of the measures obtained with rating- 
Scale techniques depends heavily upon the experience of the raters and the 
thoroughness with which they have been trained. The “halo” effect—that is, 
the tendency for an observer's general impression of the man being rated to 
influence judgments about his specific behaviors—and other forms of con- 
tamination are frequently difficult to eliminate when rating scales are used. 


from good to poor. 


Check Lists. Check-list techniques are another approach to the meas- 
urement of complex performance. A check list itemizes the specific behaviors 
which have been found to be “critical,” in that they distinguish between effec- 
tive and ineffective performance of the task (Flanagan, 1954). The observer 
then has the responsibility of detecting the presence or absence of these speci- 
fied events rather than having the responsibility for judging the over-all pro- 
ficiency level. The check list for a maintenance inspection task, for instance, 
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would list the necessary inspections which must be made to complete the ioe 
satisfactorily; and a check list devised to assess the performance of a radar 
operator would list the adjustments, identifications, and so forth, which Sep 
cate an acceptable level of proficiency. While properly constructed check lists 
reduce the amount of contamination frequently encountered in the use of 
rating scales and frequently reduce the amount of training which must be 
given to the observers, it is often difficult to devise checklists that are suffi- 
ciently elaborate and comprehensive to fully reflect all aspects of job contr 
petence. Description of the techniques and procedures used to construct rating 
scales, check lists, and similar instruments 
difficulties in their use, can be found in Guil 
(1955), and other sources, 


. as well as the advantages OT 
ford (1954), Ghiselli and Brown 


Infrequent Events as Proficiency Measures 
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those occurrences for the all success of the System. Some events simply 
occur too infrequently, or their effects on system performance are too slight, 
to be included in a test. Rare events which are critical to the success of the 
system, however, deserve more careful attention, The use of data on infre- 
quent occurrences for proficien 

upon records of performance systematically compiled over long periods ° 
time. There is often a tendenc i 
exclude unique occurrences fo they are nonrepetitive- 
However, it is possible that a i 

high rate or Proportion of uni 
missile systems, for example, r; 
than once during their devel 
malfunctioning part is rede 


2 : : uating instrument would sample skill ^s 
dealing with new problems which Tequire generalization from previous knowl- 
edge of principles, and thus would n P 


—Robert Glaser and David J. Klaus - 435 


routine tasks. For purposes of measurement, unique or infrequent events 
must be considered as a class of occurrences, even though there may be little 
essential similarity among them. 


Simulation for Proficiency Measurement 


The behaviors which have been established as the product of training 
can be made available for observation and evaluation through the use of 
Simulation techniques which utilize equipment designed to elicit joblike 
behaviors under controlled conditions, so they can be suitably measured. Sim- 
ulation is widely used in conjunction with training, as has been noted in 
Chapter 10. However, simulators developed for use in proficiency measure- 
ment are not necessarily identical with those developed for use in instruction 
(Gagné, 1954). A proficiency-measurement simulator is designed primarily 
for the purpose of eliciting certain behaviors under conditions which facilitate 
their assessment; hence, it consists largely of the instrumentation necessary 
to obtain quantitative measures of performance. A simulator suitable for train- 
ing, on the other hand, is designed primarily for the purpose of modifying 
behavior and therefore must incorporate guidance and feedback features 
Which are unnecessary if only measurement is involved. 

Whenever the behavior to be evaluated can be readily elicited for 
Measurement without special equipment, a satisfactory proficiency measure 
can be developed without resorting to extensive simulation. When criterion 
Performance is not readily accessible for measurement, however, a proficiency 
test must not only sample situations requiring the behavior, but must simulate 
these instances as well. Much of the ingenuity required during proficiency 
test development is directed at making otherwise inaccessible behaviors occur 
at the time and in the form which allows satisfactory observation and meas- 
urement, The use of simulation in assessing proficiency will be described 


further in a later section of this chapter. 


E SAMPLING AND THE RELATIVE IMPORTANCE 
OF PERFORMANCE COMPONENTS 


Sampling of Performance Objectives 


accurately reflect the objectives of 


The content of a proficiency test must 
The adequacy of a profi- 


training or some other standard of performance. 
Ciency test depends upon the extent to which it satisfactorily samples the 
Universe of behaviors which constitute criterion performance. In this sense, 
a test instrument is said to have content validity; the greater the degree to 
Which the test requires performance representative of the defined universe, 
the greater is its content validity. For many kinds of tasks, proficiency meas- 
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nized as important in task proficiency. Perhaps the most common example of 
this type of error is the emphasis usually given to the measurement of job 
knowledge and theory in instances where this information may not be rele- 
vant to actual task performance. Also, it is often assumed that certain skills 
are not sufficiently important to proficiency to be included in a test, when 
actually they represent vital aspects of job success. A radar operator prob- 
ably needs to know little about the electronic principles involved in the design 
of his equipment's circuitry; on the other hand, it may be important that he 
knows the proper way to clean the face of his display. The omission of perti- 
nent job elements may be a function of selecting too few instances of the 
universe of job behaviors to adequately represent all important aspects of 
job performance. For purposes of a stable measure of proficiency, the sample 
must not only be large enough to be representative but, as will be discussed 
Subsequently, must include a sufficient number of observations to result in a 
reliable score for a given individual. 


Weighting Component Aspects of Performance 


Closely related to the problem of sampling a universe of behaviors is the 
Problem of appropriately weighting each of the observations to be used. 
Certain aspects of behavior, of course, are of greater importance in deter- 
Mining job success than others. For example, the success which a gunner 
achieves in hitting target aircraft is probably more significant than his ability 


to carry out detailed maintenance on his weapons. Improper weighting of 


Various aspects of performance can produce serious distortions in the assign- 


Ment of proficiency scores. . ; 
The determination of appropriate weights to be given to the behaviors 


being observed can be approached in several ways. Perhaps the most com- 
Monly employed method of weighting various aspects of proficiency is by 
the judgments of experts. This procedure is frequently employed in assessing 
Performance when definitive measures of output are not available, as in aca- 
demic performance, decision-making tasks, professional skills, and so forth. 
The quality of the weights assigned by judges depends heavily on the experi- 
ence of the experts in observing the over-all performance being assessed, their 
knowledge of the system as a whole and of the influence exerted by the job 
being Observed on the output of the system, and on the representativeness of 
the judges in terms of their own backgrounds, experience, and capabilities. 
The assignment of weights by judges should be based upon the opinions of a 
Number of experts, which are combined and averaged to produce the weights 
Used, 

A second procedure that can be used for assigning weights depends on 
“mpirically identifying those specific aspects of performance which lead to 
System success or failure. The frequency with hich some particular behaviors 
are noted as contributing to or detracting from system excellence can be used 
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the criterion for weighting proficiency measures occurs when it is possible to 
identify the cost to the system, in dollars, time, or other cost measure, of 
various components of proficiency in some particular task. 

A fifth way in which several aspects of proficiency can be weighted is 
with reference to the variability in proficiency that can be observed for any 
job component. If the measures of several aspects of proficiency are com- 
bined by simple addition, the resulting score will be weighted in proportion 
to the variability associated with each of the submeasures. If the scores on 
measure A range from 15 to 25, for example, and the scores on measure B 
range from 0 to 40, a sum of the scores obtained by individuals on these two 
measures will far more heavily reflect the aspect of proficiency indicated by 
measure B than by measure A. It is reasonable to suppose that when the 
observed fluctuations in one job component are greater than the variations in 
the proficiency at another job component, the weighting that occurs as a 
function of adding scores from these two components is reasonably meaning- 
ful. Efforts should be made to independently evaluate the appropriateness of 
Such weightings, however, since it may be that some of the tasks for which 
substantial variability in proficiency can be observed do not actually con- 
tribute to job success. Furthermore, variability in proficiency scores may bea 
function of the situation in which proficiency is assessed rather than a func- 
tion of an individual's competence. Before permitting variability to determine 
Weights, it is necessary to rule out that proportion of the observed variability 
Which stems from bias in the measuring instrument. A considerable portion 
Of the variability observable in bomber crew proficiency, for example, is 
Probably due to conditions of weather and mission rather than to any factor 
relating to the performance of the crew members. While the variability due 
and experience of crew members should contribute 
à major weighting, that portion of the variability which can be attributed to 
extraneous factors should be controlled or eliminated by the method of 
Measurement and test administration or, if possible, by appropriate score 
Corrections. : . : 

A sixth possible procedure which can be used to assign weights to vari- 
Ous component skills involved in over-all job success is by establishing some 
Equivalence relationship among the components. One very common weighting 
Problem is that of deciding the relative importance of speed and errors in the 
Performance of a task. In assessing proficiency at typing, for example, weights 
are often assigned arbitrarily, such as penalizing the individual five words of 
" It is possible to actually determine the weight which 


Principally to the training 


Speed for each error. y 
Should be given to each component, however, in terms of total output. If, as 


'S usually the case, the material being typed must sooner or later be free of 
errors, an effective means of weighting could be in terms of the time required 
to correct each error satisfactorily. Proficiency in typing, then, would be 
Measured by the amount of time required to,turn out a perfect copy of the 
test material. Errors would be weighted in terms of the time spent by the 
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particular individual being evaluated to correct each error, and a traince 
who could rapidly correct an error would be penalized less than one who 
required much more time. Likewise, in assessing proficiency at aircraft Ly 
nery where a "near-miss" is often no more effective than having not fired at 
all, a score based on some combination of firi 
would probably not be as appropriate as one 
test targets destroyed during a given period of ti 


ng rate and relative accuracy 
based solely on the number of 
me. 


g ights to these samples relative to their importance to 
the over-all job and thus to the output of the s 


troduced during the sampling of behavior to be in- 
cluded in the proficiency measure can be regarded as a special instance of 
assigning weights to specific job elements, As Brogden and Taylor (1950) 
determination of 


weights have been assign 
weighting. 


[n] 


urement, there are also potential biases Which 
making proficiency measurements, These biases may affect performance 
» in which case they may be 
Y may affect scores in some 
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In general, these two forms of bias affect the degree of confidence that can be 
placed in an obtained score as being representative of an individual's level 
of proficiency. Even when obtained scores have been precisely measured, 
there is a need to consider the relevance of the score with respect to the 
standards of reference being employed. This is the problem of the validity of 
a proficiency test. The degree of validity in a proficiency measure is a func- 
tion of the difficulties which exist in identifying, quantifying, selecting, and 
weighting the behaviors to be assessed. Nevertheless, procedures are available 
for investigating the validity of proficiency measures and for increasing their 
relevance with respect to the requirements of a particular system. 


Contamination and Systematic Measurement Bias 


Several sources of bias which occur in the process of measurement and 
Which tend to exert a systematic influence on estimates of proficiency can be 
identified. One of these is the error in measurement which results from 
assessing proficiency within an operating system. Since the observations to be 
made, or the scores to be obtained, are frequently determined to some extent 
by the performance of other components in the system, care should be taken 
to control the influence of such biasing influences during measurement. For 
example, Thorndike (1949) describes several biasing factors which might 
influence the evaluation of a bombardier during an airborne exercise. He 
Points out that the accuracy of a particular bomb drop may be a function of 
the turbulence of the air, the condition and calibration of the bomb sight, the 
accuracy in construction of the bomb, its freedom from physical defects, the 
Skill and conscientiousness with which the pilot flew the plane, circumstances 
of time of day, altitude, or length of bomb run under which the bomb was 
dropped, as well as the skill with which the bombardier performed his task 
(p. 129). Sources of bias such as these can be reduced by careful control of 
the test situation as, for instance, by effective simulation or careful adminis- 


trative arrangements. 


Further sources of contaminatio a : l 
tion of the test instrument. An examination may differentially affect the scores 


of individuals because of factors unrelated to actual job performance. The 
test may lean heavily upon the individual's skill in understanding test direc- 
tions, for example, or his speed in reading lengthy descriptive passages. In 
these instances, the influence verbal facility may have on the test score is to 
r n such a way that they systematically penalize 
aries and reading skills, and thus fail to ade- 


quately reflect their proficiency in tasks which do not depend directly on 
Verbal skills. A proficiency test may also be prepared M such a way as to 
ifferentially promote response sets in the individual being evaluated (Cron- 
pach, 1946). A response set is a general tendency to respond consistently 
In one particular way. Depending on the way in which it is constructed, a 


n are sometimes found in the construc- 


distort proficiency estimates i 
Individuals with poor vocabul 
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test may encourage some of the individuals taking it to be overly piu 
their work, for example, and thus receive a higher or lower score than m g 
i € been the case. ; 

aie cene individuals are used to evaluate proficiency on the basis of 
their observations of performance, as is the case when ratings or judgments 
are made, several kinds of judgmental errors can occur. M 
of bias have been studied in detail (Guilford, 1954; Ghiselli and Brown, 
1955). In general, this source of contamination stems from some previous 
knowledge or experiences on the Part of the individuals used as judges. For 


example, the judge may have acquired a steretoyped belief that people with 
red hair are erratic and impetuous in their 


are apt to be more thou 


tendency such as this causes the judge to rate certain individuals higher or 
lower than should be the case on specific tasks—a finding, as mentioned 
“halo” effect, Similarly, a particular 
Strict in his judgments and thus give 
iduals he observes, Then again, raters 
umber of individuals as average. Prior 
dividual's previous performance may 
ments made during proficiency assess- 


any of these sources 


may tend to judge a disproportionate n 
knowledge by the raters about an in 
also have a substantial effect on judg 


All in all, these judgmental biases are mo: 


st pronounced when the judg- 
ments themselves are fairly general in Scope 


or refer to poorly specified 
trol over judgments Sometimes may be obtained by 


bservers. Frequently, it is also possible to overcome 
arefully focusing the aS: d 


carefully training the o 
these biases by more ¢ 


affected only when systematic 
he scores of Particular examinces, as iS 
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Reliability and Random Measurement Errors 


Aside from instances of systematic bias that may occur in the course of 
proficiency measurement, sources of unsystematic error may have an influence 
on the precision of the measures that are obtained. The effects of unsyste- 
matic error can be understood by considering an individual score as one 
instance of a large number of measurements. If all the measurements were 
perfectly stable—that is, if there were no unsystematic errors influencing the 
scores—an individual would receive exactly the same score each time his 
performance was measured. In actual practice, however, repeated measures 
of performance yield a distribution of scores rather than a single estimate of 
Proficiency. The extent of the variability in scores is dependent on the pro- 
Portion of the score attributable to unsystematic error. The greater the dis- 
Persion of obtained scores when a number of observations are made, the less 
the reliability of the measure. In effect, then, reliability refers to the precision 
of measurement; when the dispersion of scores all representing measures of a 
Single individual on a particular task is large, the measure lacks reliability, 
and the test will be incapable of differentiating consistently among individuals 
Who are at different skill levels. If the measure is precise—that is, if the 
Scores are nearly identical—the measure is reliable in the sense that the 
range of values which can be expected for any one individual is small. 

When an individual's proficiency is assessed, it is important to have some 
indication of the degree of reliability for the score that is obtained. For 
Practical reasons, it is generally not feasible to repeat the measurement proc- 
ess often enough to obtain a large number of test scores. In practice, an 
estimate of the reliability of an obtained score can be made in two different 
Ways, both of which involve the comparison of scores obtained from two sets 
Of measurements on a single group of individuals. One procedure, called the 
lest-retest method, involves correlating the scores obtained when the same 
lest is given on two separate occasions. The second procedure, called the 
alternate-form method, involves correlating the scores obtained from two 
different but parallel forms of the same test, administered at the same time. 
A frequently employed variant of this second method is the "split-half" pro- 
cedure, which considers the two halves of a test as separate forms, so that 
the scores obtained from the two halves may be correlated. It is also possible 
to combine the two procedures by evaluating performance using alternate 
test forms administered on two separate occasions. The correlation coefficient 
computed from the two sets of scores, whether the test-retest or alternate- 
Orm procedure is used, is called the reliability coefficient. ; 

The two methods for arriving at reliability coefficients each require a 
Slightly different interpretation (Cronbach, 1960; Thorndike, 1949). When 
test-retest procedures are used, the reliability coefficient indicates the sta- 
bility of a score over a period of time. Herce, it is sensitive to sources of 
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unsystematic error which are a function of day-to-day fluctuations in the 
individuals being tested and random fluctuations in environmental influences. 
When alternate-form procedures are used, the reliability coefficient indicates 
the equivalence of scores representing different samples of test items. This 
procedure is especially sensitive to sources of unsystematic error which are a 


function of the representativeness of the particular observations of behavior 
being made during proficiency assessment. 


Sources of Unreliability. One way to look at possible specific sources 
of unreliability is to consider the conditions which may produce differences 
in proficiency test scores when à single test is administered a second time 
or when an alternate form of a test is used to provide a second measure of 
an individual's performance level. Unequal scores from the two measures will 


other, the two measures are nonequivalent 
performance observed. One major source 
situation is a function of variations in the 
s being assessed. The weather, for instance, 
a pilot or radar operator in some unsys- 
ay, other situational factors such as noise, 
ctions, and other environmental influences 
domly affect test scores. 

bility is a function of fluctuations associated 
ion stem in which an individual's performance is being 
measured. Variations in Scores due to system instability may reflect random 
nical components or in other human compo- 


ance of a crane operator, for example, may 
ut the condition of his equipment as well as 
his helper on the ground, 


» it is important to fre- 
andom malfunctions, so aS 
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teristics of test items which may vary from one sample of items to another 
in such a way as to produce random fluctuations in test scores are: the extent 
to which correct responses may be the result of guessing, the particular 
sequence of items, the difficulty of the items, the ambiguity in the questions, 
the degree to which examinees differentially may have been exposed to the 
item previously, and the degree of restriction on possible responses to the 
item as a result of test format. The determination of the equivalence of item 
samples used in alternate forms of a test, in terms of such statistical consid- 
erations as test means, score variability, and item characteristics, are discussed 
by Gulliksen (1950). 

The complexity of the behavior being evaluated is a fifth type of influ- 
ence on the reliability of a proficiency measure. If the behavior being meas- 
ured involves many dimensions of performance, the items included in the 
test instrument will necessarily be relatively heterogeneous. Since it is possible 
that an individual's proficiency level may fluctuate considerably from one 
dimension to the next, each component element in one sense represents à 
somewhat different test. Thus, each of these dimensions is susceptible to all 
the sources of unreliability that have been described above. As the number 
of dimensions involved becomes larger, the number of items pertinent to any 
one element of over-all performance may be decreased in proportion to the 
total test. The effect of item-sampling fluctuations on score reliability is often 
assessed by procedures which measure the degree of homogeneity among 
items; that is, the extent to which performance on all test items is correlated. 
This internal consistency among test items is generally used to indicate the 
reliability of the test. The Kuder-Richardson procedure for assessing the relia- 
bility of a test was developed using this approach (see Gulliksen, 1950). It 
is important to recognize that the use of this procedure assumes that the test 
is measuring a unitary skill. This may be an unrealistic assumption in pro- 
ficiency measurement, however, in that heterogeneity in a sample of items 
May be necessary to accurately reflect the nature of criterion performance. 

A sixth source of unreliability is attributable to the characteristics of the 

uman component itself. The performance of the individual being assessed 
Uctuates as a function of temporary variations in the state of the organism. 
Some of the factors frequently involved which may affect the stability of 
Measurement are the individual's motivation, emotional state, susceptibility 
to fatigue and stress, and variations in the individual’s speed of work and 


tendency to guess. : 
As can be surmised on the basis of 


Measurement, some degree of unreliability is in 
the proficiency of the human component in any complex system. One of the 


tasks of the test constructor is to minimize the degree to which random error 
Influences the test score of an individual and thereby increase the precision of 


at score for assessment purposes. 
Finally, reliable measurement Is 


these six sources of random error in 


dependent, to a large degree, on careful 
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control of the test situation. It is frequently possible to reduce the DO o 
instability in scores to some extent by manipulating one or more aspec : 4 
the measurement procedure. By clearly specifying the conditions under whic! 
the observations are to be made, it is possible to increase the standardization 
of the test situation and thereby reduce the amount of fluctuations in scores 
attributable to random error of measurement. 


Estimating Test Reliability. In estimating test reliability by means of a 
correlation coefficient, the actual reliability of a particular test instrument will 
be underestimated if the Tange of talent being observed is small in relation 
to the actual range among individuals on which the test will be used. For this 
Teason, statistical estimates of the reliability of a test instrument should be 


based on a reasonably representative sample of the population for which the 
test is designed. In some cases, 


are largely a function of the cha 
easy or too difficult for the pop 
on the test which actually di 


echniques are used. In a sense, then, the reliability of 2 
criterion-based score can be thought of a 


while the reliability of a norm-refer thought of as the 
consistency of a score. This distinction is sim; 

dike (1949), in which he points 
relative consistency in estimating tes 
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When test instruments are used for predictive purposes, the concept of 
validity is usually defined in terms of the relationship between test perform- 
ance and subsequent criterion performance—that is, performance at the end 
of training or on the job. Proficiency measurement, as it has been considered 
in this chapter, is concerned with criterion performance itself. Thus, the con- 
cept of validity with respect to proficiency measurement has a somewhat 
limited meaning. The validity of a test used to assess proficiency is not estab- 
lished in the sense of how well it can predict any consequent or subsequent 
behavior. Instead, the validity of a proficiency measure is defined in terms of 
the content of the test. The validity of an arithmetic achievement test, for 
example, is not established by how well it may be able to predict future 
performance, but rather in terms of how adequately the content of the test 
reflects the kinds of behavior it is supposed to measure. Only in those cases 
when the achievement test is used as a predictor can the correlation of scores 
with future performance be used to establish some measure of validity. 

A proficiency test, like all other tests, can be defined as a stimulus situa- 
tion which has been constructed to evoke the particular kinds of behavior 
to be measured or assessed. In one sense, then, the validity of a test instru- 
ment is reflected in the degree to which the behaviors evoked provide a means 
for consistently differentiating among individuals according to their perform- 
ance. In particular, a proficiency test can be considered valid if it discriminates 
among individuals presumed to range from no proficiency to high proficiency 
in a given skill. The validity of a proficiency test, then, is established by 
demonstrating that the test scores reflect differences in skill levels of the per- 


formance being assessed. 


Establishing Proficiency Test Validity. The most common method em- 
Ployed to establish proficiency test validity is to employ the judgment of 
experts in the skill area under consideration. When. judges are used, they 
typically participate in the development of the test instrument, so that the 
Validity is established as the test is completed rather than afterward. In 
building a test to assess proficiency in supervisory skills, for instance, judges 
are used to specify in detail the kinds of behaviors to be observed and the 
Skills to be measured. As the test items are prepared, the judges further 
evaluate the instrument in terms of the adequacy with which the items reflect 
the performance under consideration and encompass the range of skills 
involved. » 

While the use of judgmental methods for establishing the validity of a 
Proficiency test is quite widespread, especially in the case of academic achieve- 
ment examinations, several difficulties are often encountered. First, it is 
difficult to establish the expertness of a judge. It is possible that proficiency 
at a task does not qualify an individual as an expert in identifying the com- 
Ponent skills which contribute to task proficiency. Second, diflerent judges 
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will have different definitions of skill levels, and it may be difficult to n 
satisfactory agreement among experts, especially when the behaviors Th " 
consideration are complex. Finally, and possibly most important, it is likely 
that the judgment of experts will be more adequate with respect to the — 
age of the test than with respect to the quality of the items. Thus, the use o 
judges to assess test coverage in the preparation of item specifications is often 
more successful than the use of judges in evaluating the discriminability of 
the items. 

In many respects, a preferable method for establishing the validity of a 
proficiency test employs the use of experimental techniques (DuBois, Teel, 
and Petersen, 1954). Using this approach, proficiency is viewed as the prod- 
uct of prior training and experience. Thus, the scores on a proficiency test 
should adequately reflect differences in performance levels attributable to 
these previous conditions. An individual who has completed a course in map 
reading, for example, should demonst 


Pared to measure this skill th 


designs can be employed to demonstrate test 
area. F 
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An important consideration in establishing the validity of a proficiency 
measure is the relationship between the skills being assessed and ultimate job 
requirements. The proficiency test used at the completion of a training course, 
for instance, may establish the degree to which an individual has acquired 
knowledge about his job or has mastered the necessary skills. It does not 
demonstrate that the individual will perform effectively in the job situation. 
An individual with generally proficient skills may turn out to be an occupa- 
tional failure because he spends a considerable amount of time on his job in 
a wasted manner. Whenever such a possibility exists, a valid proficiency-test 
instrument should sample job behaviors in such a way as to diflerentiate 
individuals who will perform a given task satisfactorily from those who are 


also able to perform it but will not. 


Validity Relative to Reliability. A problem sometimes confronting the 
developers and users of proficiency measures is the evaluation of the relative 
merits of a test instrument which has satisfactory validity but low reliability 
versus one which has poor validity but high reliability. The validity of a 
Proficiency test is primarily a function of the accuracy with which the task 
has been analyzed and the skill with which the items have been selected. 
Thus, the validity of a test is limited by psychological knowledge about the 
task and how this knowledge permits the task to be analyzed into component 
behaviors. The reliability of a measure, on the other hand, is influenced both 
by the degree to which psychological knowledge enables the precise deter- 
Mination of those factors which are apt to influence performance and by the 
Control of factors in the test situation and test instrument which may influ- 
ence test scores. Both the validity and reliability of a proficiency measure are 
limited by the extent to which psychological knowledge concerning the task 
is complete enough to specify the behaviors to be assessed, and the means by 
Which accurate measures of performance can be obtained. Reliability is also 
à function of sources of error which can be controlled by careful design of the 
test instrument and careful administration practices. As a consequence, it is 
Benerally less difficult to improve the reliability of a test instrument than to 
raise its validity. For this reason, it is often more efficient to attempt to 
improve the reliability of a test with promising validity rather than to retain 


a highly reliable test which has lower validity. 


In general, this section has dealt with several of the considerations 
involved in the construction of any psychological test. The problem of syste- 
Matic measurement bias, random errors, and test validity are n Tare in 
Proficiency assessment as they are in aptitude ccr p T eae pw p 
Measurement problems in proficiency testing, where L Panem ehavior 
Itself, rather than a prediction of criterion behavior, Is O e concern, 
I5 somewhat different from aptitude testins- ee iat rate 

tas are especially critical in proficiency measurement, as a result of the field 


450 Proficiency Measurement 


environment in which many of the tests are administered. Furthermore, the 
relatively frequent use of rating scales in proficiency measurement causes these 
measures to be highly dependent on observer judgments, and consequently, 
many proficiency tests lack the potential precision of more objective measures 
of performance. Establishing the validity of a proficiency measure is likewise 
different from the procedures used to determine the predictive validity of an 
aptitude test. In proficiency tests, validity is determined by demonstrating the 
effects of manipulations which are thought to influence proficiency levels. 


[5] 


ELICITING BEHAVIOR FOR MEASUREMENT 


A psychological test can be defined 
behaviors by means of s 
pointed out previously, 
ticular situation are rep 
the validity of a test 


às a technique for evoking particular 
pecially designed stimulus situations. As has been 
the degree to which the responses evoked by a par- 
resentative of the defined terminal behavior indicates 


developed to evaluate proficiency. Furthermore, the 
degree to which the responses evoked are recorded and evaluated in a con- 


sistent and unambiguous manner determines test reliability. These factors, in 


turn, are dependent on the characteristics of the stimulus situation and the 
means employed to score or assess responses, 


"recognition tests" OT 
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tests may vary will be fore looking at some representative pro- 
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The Dimensions along which Proficiency Tests Differ 


Process versus Product. 


other instances, tangible produc 
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discoveries made or the importance of these discoveries. In many situations, 
however, it may be more practical to measure the proficiency of the scientists 
by observing behaviors which are likely to lead to important discoveries. 
By measuring the process, it may be possible to infer the product. The scien- 
tist's skill in identifying problems, collecting evidence, and analyzing data are 
some of the processes that can be measured. 

When products are used as the basis of measurement, long-term as well 
às immediate products may be included in the sample. Long-term products 
frequently reflect aspects of proficiency which are difficult to measure in any 
other way. Supervisory behavior, for example, can be at least partially assessed 
in terms of employee turnover in a given department or division. Presumably, 
differences in the rate of turnover between two comparable departments 
reflects differences in employee morale and, in many cases, can be inter- 
Preted as indicating the kind of supervision the employees have received. 

Categorizing proficiency tests as to whether they measure process or 
Product depends primarily on where the observations, and hence the evalu- 
ation procedures, are focused. If primary attention is given to output, the test 
Measures product; if attention is given to the means, the test measures proc- 
ess. Whether process or product is used as the basis for proficiency measure- 
ment often depends on which is most accessible and which is most likely to 


Produce a reliable measure of performance. 


Necessity for Observer Judgment. Proficiency tests can also be dif- 
ferentiated according to the degree to which a test score depends on observer 
intervention. Some test instruments involve only minimal observer judgments 
While others depend heavily on them. In generai, the distinction between 
Objectively scored and subjectively scored tests reflects the degree of relia- 
bility that can be attained in assessing performance. Proficiency tests which 
are scored primarily on the basis of the presence or absence E ignis require 
less observer judgment, and hence are more objective, than tests which 
depend on observations requiring extensive interpretation by a judge. In 
evaluating aircraft maneuvers, for instance, some aspects of the pilot's per- 
formance can be assessed by observing the aircraft’s instruments. These 


Observations are more objective than other aspects of each maneuver, which 
: to its tightness, its smoothness, and 


are z cing judgments as 

So &ni e Feros d riji to improve the reliability of a measur in 
Which observers are used by reducing the extent to which judgments are 
required. For example, leadership or supervision generally can ial ^ 
More reliably by means of check lists than by the use of rating scales. The 
Tole of observer judgment in proficiency measurement is especially clear in 
educational testing where multiple-choice and other objectively scored tests 
are used mainly because they require considerably less human intervention 
than Subjectively scored essay tests. ’ 
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Component versus Global Measures. Some proficiency-measuring in- 
struments attempt to assess performance of component tasks separately, while 
others attempt to evaluate job proficiency in a more global sense. Each of 
these approaches has its advantages. Whenever component skills are assessed, 
the results can be used for diagnostic purposes—that is, to indicate the par- 
ticular tasks at which any given individual is highly or poorly qualified. A 
global test of proficiency, on the other hand, is more likely to sample virtu- 
ally all interacting aspects of over-all job performance and weight each 
component skill in terms of its contribution to general job proficiency. As an 
illustration of this difference, a radio repairman's job performance can be 
evaluated by means of a series of tests based on components of his job such 


as theory, parts identification, troubleshooting, and repair procedures. His 


performance also can be assessed using a global approach, by basing the 


measurement on his over-all skill in identifying and repairing a selected group 


of troubles in which all the relevant component behaviors are required. 


Observational Standardization. A fourth dimension along which pro- 
ficiency tests may differ is related to the degree to which proficiency can be 
measured in a real-life situation. In many instances, it is necessary to modify 
the job situation in order to make precise measurement possible. For one 
reason or another, the job situation or the nature of the task being assessed 


eral, the degree to which performa 
of extraneous variables determin 
must be made “artificial” for pu 
observation. The task also may require modification before accurate meas- 
urement of performance is possi administrative skill, 
for example, it may be necessar peiie in the test 
situation to a discrete number r: an permitting the very large range of 
responses possible in the actual job. 
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but it does not require that the responses be made under circumstances iden- 
tical with those that exist when jumping from an aircraft. Similarly, the 
proficiency of student nurses in using a hypodermic syringe can be assessed 
by asking them to inject a grapefruit; while the responses the nurse must make 
are quite close to those which must be made on the job, the stimulus settings 
are substantially different. 

There are other test modifications that can be made which will tend to 
influence the form of the response rather than the stimulus situation. Examples 
of changes of this kind are tests which require multiple-choice responses 
rather than composed answers to measure educational achievement, tests 
which measure ability to identify tools as an indication of tool usage on the 
job, or tests which require the recognition of safety hazards as an indication 
of an individual’s proficiency in utilizing safety practices and observing pre- 
cautions. 

In general, the extent to which stimulus or response realism is modified 
for purposes of proticiency measurement depends on the degree to which 
Satisfactory test validity and reliability can be achieved, by testing equipment 
and situations which are in keeping with the feasibility and cost considera- 
tions pertinent to the construction and administration of the test. 


Types of Proficiency Measures 


The tests available for specific proficiency-assessment purposes cannot 
be consistently categorized. Tests may vary according to their degree of 
remoteness from the job situation and the degree to which the behaviors that 
are elicited differ from terminal behaviors. Furthermore, tests can differ 
Widely with respect to the cost of their administration, the degree of control 
€xerted over extrancous factors and influences, the measurement procedure 
used, and the nature of the behavior contributing to adequate performance. 
It is not surprising, then, that the terminology traditionally employed in 
describing psychological tests, and proficiency tests in particular, tends to be 
inconsistent and reflects a variety of test characteristics. The term perform- 
ance test,” for example, often means a test involving motor-manipulative as 
Opposed to verbal skills; a term such as “procedural test identifies a pro- 
ficiency measure which assesses an individual's pertormanes in relating and 
Ordering job steps; and a term such as “group test” refers to a test instrument 
Which can be administered to more than one individual at one time. 

Despite this lack of consistency, proficiency tests may be grossly cate- 
Borized on the basis of their remoteness from actual job performance. This 
remoteness may take the form of changes in the behavior elicited for meas- 
urement or in the eliciting stimuli themselves. In most instances, though, as 
the stimuli become more remote from those found in the actual job situation, 
the responses elicited are likewise less similar to those found in job perform- 
ance, At one extremę along this continuum of remoteness is the measurement 
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of proficiency during actual job performance. At the other extreme are meas- 
ures which are not obviously similar to the criterion task, but instead assess 
performance at tasks which correlate with on-the-job behavior. Between 
these two extremes are test situations which attempt to simulate the job task 
while at the same time offering effective control of the factors which in “real” 
situations are likely to influence measurement. Although it is clearly a con- 
tinuum of remoteness which is involved in categorizing proficiency measures 
in this way, it is possible to identify the three major segments of this con- 
tinuum. These are on-the-job measures, simulated-performance measures, and 
correlated-behavior measures. 


On-the-Job Measures. Ideally, the measurement of proficiency should 
be accomplished during typical performance of the system and under condi- 
tions generally present during day-to-day operations. As noted earlier, how- 
ever, the degree of control which it is possible to achieve in a job situation 
is generally less than satisfactory for obtaining accurate measurements. Fur- 
thermore, attempts to standardize the situation for proficiency-measurement 
purposes, and even the observational or recording processes themselves, fre- 
quently introduce considerable artificiality into the situation. The feasibility 
of accurate measurement during typical system operation also may be ? 
problem, in that work spaces are rarely designed to facilitate observation. 
Despite these difficulties, however, attempts to secure reliable and valid 
measures of on-the-job performance have been carried out successfully. 

The techniques which have been employed for on-the-job proficiency 
assessment generally involve controlled observations of specified attributes 
of over-all performance or product. Gross judgments based on free observa" 
tion of performance generally result in unstandardized and hence unreliable 
assessments, in that the behaviors attended to and the basis on which they 
are judged vary from observer to observer. On the other hand. procedures 
which specifically direct the attention of the observer toward critical aspect? 
of task performance or product quality, and which include definitive standards 
to be used as the basis of judgment, can be employed satisfactorily in on-the- 
job situations. Since all aspects of the performance or product cannot be 
evaluated at one time, it is necessary to employ techniques such as time 
sampling, recording devices, observational instruments and other means to 
guide the observations being made. Many procedures have been devised for 
this purpose. For example, it is possible to measure product samples whic 
have been selected so as to be representative of an individual's output. Chee 
lists are used which indicate the procedural Steps necessary to accomplis 
a given task or the presence or absence of certain desirable behaviors. per- 
formance records can be developed which facilitate the recording of incidents 
of outstandingly effective or ineffective behavior over long periods of time: 
Finally, it is possible to use rating scales which have been carefiily construct? 
and validated. 
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. An interesting example of proficiency measurement in an on-the-job 
situation is described by Siegel, Richlin, and Federman (1958) Their ai 
was to assess the performance of parachute riggers some nmm after miris 
was completed. In this case, attempts to evaluate product quality as " i di 
cator of proficiency did not prove feasible and, instead “Technical Behi * b 
Check Lists," designed to assess individual task performance pst devel vem 
On the basis of curriculum outlines, discussions with school desear at 
direct observation of students during training, a list of tasks son ae 
and reviewed by field personnel. Items were then selected for aes i 
the final check list if they referred to tasks which occurred relatively fre- 
quently, were judged to be critical to over-all job performance, and reflected 
the existence of a range of proficency as indicated by the range of supervision 
required for individuals at different proficiency levels. The final verion of 
the check list directed the observers’ attention to such specific tasks involved 
an the parachute rigger's job as "inspecting and checking continuity of lines," 
cutting material," "sewing pieces together," "testing quick release latch," 


and "fitting crash helmets" (see Figure 12.1). 
An example of the use of tangible product measurement to assess pro- 


ficiency is that reported by Mackie and High (1956). In this study, a typical 
Navy Machinist Repairman product, a valve stem and hex fitting, was manu- 
factured in an on-the-job setting as the standardized proficiency-measurement 
task. The problem was presented in such a way as to be typical of actual 


Degree of supervision required 
when months in squadron was 


Job assignments 
(sample items) 


A. Laying out measurements on material 


B. Sewing pieces together 


C. Rigging container and harness of seat-type parachutes 


D. Whipping and folding canopy 


E. Checking and installing barometric releases 


F. Attaching para-raft to parachute 


G. Lecturing on survival equipment and techniques to 
pilots, aircrewmen, and plane captains | 


An on-the-job checklist for the task of parachute rigging. 
ped for many kinds of tasks in a variety 
d Federian, 1958; courtesy Applied 


Figure 12.1. 
Such check lists have been develo 
of jobs. (After Siegel, Richlin, an 
Psychological Services.) 
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product requirements. The component produced was evaluated by means of 
33 separate inspections. Of these, 16 were objective scores based on meas- 
urements made with rulers and calipers. The remaining 17 measures were 
more subjective and consisted of judgments of fit, finish, and alignment of 
the valve stem and fitting. Correlation coefficients representing agreement 
among the judges were found to range between .70 and .98 for the objective 
measures and between .71 and .99 for the subjective measures. 

When on-the-job proficiency measures are used, it is often important to 
collect pertinent information on the individual's performance over an €x- 
tended period of time. Such a procedure tends to increase the stability of 
evaluations by decreasing the influence of day-to-day fluctuations in working 
conditions. An example of a proficiency measure prepared to facilitate such 
a longitudinal assessment of proficiency is the Clinical Experience Record 
for Nursing Students (Flanagan et al., 1960). This performance record was 
developed on the basis of detailed information obtained on the effective and 
ineffective performance of nurses in on-the-job situations. A total of 2073 such 
incidents were collected from supervisors, nursing instructors, and patients 
An analysis of these incidents defined a list of twelve categories under which 
observations could be recorded, such as “Meeting the patient's adjustment 
and emotional needs" and "Relations with co-workers, physicians, and visi- 
tors." These categories and their accompanying examples direct the attention 
of the observer to the kinds of behaviors most likely to differentiate between 
proficient and nonproficient nurses. A performance record form such as this 
one is designed for extended use, during which time records are made O 
behaviors to be encouraged, such as “Noted inconsistency in medication. 
treatment, or diet order," and those needing improvement, such as “Cause 
patient discomfort due to lack of skill or awkwardness.” After each recording 
period, which might be several months, the compilation of specific instances 
serves as a basis for documenting evaluations of proficiency. 

As these examples illustrate, on-the-job measures generally represent 
little alteration from the real job situation. Most of the efforts of the test 
constructor are devoted, not to the preparation of stimulus material, but tO 
developing suitable procedures for making and recording the necessary 
observations. Perhaps an ideal approach to the evaluation of proficiency dur 
ing system operation would be to design effective performance measurement 
and recording instrumentation to be incorporated into the system at the time 
it is constructed. This approach is similar to the one used during the develop“ 
mental stages of missile design, where early models include sufficient tele- 
metering equipment to provide accurate feedback as to the functioning ° 
each important component during the launching and flight of the missile. 
The parallel use of telemetering devices to assess human performance wou 
provide valuable information which could be used to strengthen training an 
human engineering efforts during system development. 
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Simulated Performance Measures. While on-the-job measures are fre- 
quently viewed as potentially the most valid for assessing proficiency, the 
conditions present in the job situation are such that the uncontrolled sources 
of error which may arise frequently tend to overshadow differences in pro- 
ficiency among the individuals being evaluated. As a result, the job situation 
must be simulated in a controlled manner so as to effect a standardized 
environment in which performance can be readily and reliably assessed. The 
development of simulated job situations is often an expensive and inventive 
procedure. The essence of test simulation is the design of test stimuli which 
will evoke joblike behaviors that can be objectively measured. 

The general category of simulated performance measures include a 
variety of proficiency-test techniques. Some of the most frequently used 
simulated measures are performance tests, in which an individual's proficiency 
is evaluated with respect to simulated system equipment and simulated en- 
vironmental conditions. Another kind of simulated measure is the situational 
test, in which proficiency is assessed by having the individual attempt to act 
à realistic role in a "staged" situation. Finally, there are component-skill 
Measures, where only a portion of over-all job performance is elicited by 
simulating the requirements of a specific subtask. This latter procedure is the 
Case when a pilot is tested on emergency procedures, the electronic technician 
is tested on his ability to read schematic diagrams or use instruments, ora 
Machinist is assigned the task of setting up his lathe in a particular way for 
evaluation purposes. " 

When the measurement procedure involves the operation of actual Sys- 
tem equipment in a controlled test situation, only minimum simulation of 
actual job conditions may be required. Thus, evaluating the skill of a typist by 
having her prepare a copy of a letter, or evaluating the skill of a radar oper- 
ator by measuring his ability to identify patterns projected onto E» i: 
involves only a limited amount of ingenuity on the part of the test constructor; 
and the more challenging aspects of the problem may be those of rag 
Appropriate job conditions and requirements for inclusion in the hrs en 
actual system facilities are not available for assessment [iin i S neces 
Sary to employ various methods to elicit the specific be sea to e near 
ured. In these cases, the more critical characteristics of the equipment an 
Materials to be used must be suitably simulated in addition to the job environ- 


Ment, a ja 
An example of a proficiency test procedure developed to elicit adminis- 
trative ‘date is the In-basket Test for school administrators (Fredricksen, 
1961) This test. which involves extensive simulation of representative aspects 
, x» th of an earlier version prepared for 
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is to interrupt normal system operation so as to achieve a controlled test 
situation which nevertheless involves actual equipment. “War games" and 
battle exercises are examples of this type of procedure. The use of such vast 
amounts of equipment for proficiency measurement purposes is not always 
practical, however. As an alternative, it is frequently possible to develop 
simulated situations which not only facilitate measurement, but which involve 
relatively lower costs. For example, it is frequently possible to use training 
simulators for proficiency-assessment purposes, as was the case with the 
miniature tank battlefield. However, it is to be cautioned that training equip- 
ment often includes features to assist learning which are not included on the 
job, and these must be taken into account. The use of system simulation 
techniques in measurement is further elaborated in Chapter 13. 


Correlated-Behavior Measures. An extreme position along the dimen- 
sion of correspondence between the conditions present on the job and in the 
proficiency test is represented by tests measuring correlated behaviors. of 
the three types of proficiency measures that have been discussed in this 
section, correlated-behavior measures are the most remote from the actual 
job situation. In general, these tests represent a degree of abstraction from 
skill components required as part of proficient job performance. The most 
widespread type of correlated-behavior measure is the elicitation of verbal 
responses to assess skills which are substantially not verbal. Examples of this 
type of proficiency measure are the tests of job knowledge, vocabulary, and 
nomenclature used to evaluate performance at procedural and manipulative 
tasks. Other types of correlated-behavior measures are those which involve 4 
deliberate modification in response topography so as to facilitate the record- 
ing and evaluation of responses. A common instance of this kind of construc- 
tion is the use of multiple-choice tests to measure the ability to produce 
appropriate responses by measuring ability to recognize them. Another exam- 
ple of the use of correlated behavior measures is the measurement of related 
events, such as income or number of promotions, as a measure of skill level 
attained. The use of related events is somewhat analogous to the evaluation 
of product as a means of assessing on-the-job proficiency. However, the rela- 
tionship between job performance and associated events is not always clear. 
and evidence of the correlation between them must be established prior to 
their use as proficiency measures. 

An example of the use of correlated-behavior measures to assess prO- 
ficiency when more direct procedures would be unnecessarily dangerous or 
expensive is the test of safety precautions for personnel aboard submarines 
developed by Wilson and Mackie (1952). Their test consisted of a series of 
15 photographs which depicted men performing routine tasks aboard sub- 
marines. Examinees were required to indicate whether or not safety violations: 
such as the failure to wear gogg'es, were evident in the pictures and, if so. to 
list them. The validity of this test was demonstrated by data indicating a rela- 
tionship between test scores and amount of training and experience. The use 
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of correlated behavior in this instance proved satisfactory, especially con- 
sidering the low frequency of safety violations and the difficulty which might 
have arisen in trying to observe them during job performance. It should be 
noted, however, that being able to respond verbally to safety violations does 
not insure that such practices will be carried over into actual performance, and 
some discrepancy between what a man knows and what he does can be 
expected. That caution must be exercised in interpreting the results of pro- 
ficiency assessment based largely upon verbal descriptions can be illustrated 
by the example of the parachute rigger. One certainly would feel more secure 
after observing a rigger pack parachutes in a job situation than he would if 
he knew only the rigger's proficiency in describing verbally the procedure he 
would follow. 

Because they are easily constructed, verbal or paper-and-pencil tests 
of job knowledge are frequently used to evaluate an individual's proficiency. 
When such measures are employed, it must be assumed that the job knowl- 
edge or theory being assessed is relevant to actual job performance. Tests 
measuring knowledge of technical information, tool nomenclature, technical 
vocabulary, or underlying theory do not measure actual performance. Instead, 
they measure verbal knowledge about the job and, hence, assess correlated 
behaviors especially in those cases where the job depends on motor and 
manipulative skills. The degree of correlation between tests of job knowledge 
ànd actual job performance is apparently related to the amount of perceptual 
and motor practice required for skilled performance, and the extent to which 
Verbal practice has accompanied instruction In the motor task. An example 
Of a test of job knowledge which was successful in differentiating among pro- 
ficiency levels is reported by Hill, Buckley. and Older (1952). Their paper- 
and-pencil test, which included sections on job information, troubleshooting, 
and tool knowledge, correlated .63 with ratings on proficiency in a job sample 
involving representative tasks carried out with actual equipment. Thus, it is 
Possible to construct tests which measure job proficiency indirectly by assess- 
ing the individual's knowledge about proper performance. These paper-and- 
Pencil tests have the advantage of being easily constructed and economically 
administered, as compared with on-the-job or simulated-performance meas- 
ures, |t should be pointed out, however, that many more variables than job 
knowledge can affect actual performance, and before such tests are used for 
üssessment, the degree of correlation must be assessed empirically. 

The measures of correlated behavior discussed thus far have been con- 
Cerned with aspects of an individual’s performance which involve abstractions 
from behavior observable in job situations. The development of these meas- 
ures is predicated on the correlations of various component behaviors in the 
Performance of a given individual. Job performance is also correlated with 
*nvironmental events, either directly in terms of job product, or indirectly in 
terms of events affected by an individual's output. Examples of this type of 
were investigated by Turner (1960), who compared a 
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performance. The objective measures used were employee grievances, vol- 
untary turnover, absences, suggestions, occupational injuries, disciplinary 
actions, amount of scrap, excesses in the use of tools and materials, and effec- 
tiveness as measured by time to complete a job compared with established 
time allowances. These correlated-event measures of proficiency were found 
to have low correlations with less objective ratings of job performance. Thus, 
Turner found little relationship between objective correlated-event measures 
and production-foreman ratings which assessed similar areas of job perform- 
ance. Despite this lack of correspondence, the approach of using correlated- 
event measures deserves further study, since they may well be more repre- 
sentative of supervisory proficiency and effectiveness than are more subjec- 
tive ratings. 

An important characteristic of correlated-behavior measures is that thcy 
frequently include the modification of the topography or form of a response 
so that it may be objectively and easily observed, recorded, and scored. AS 
has been indicated, the responses required on a test are frequently in multiple- 
choice form, even when the job involves recalling and constructing a response 
instead of recognizing the proper alternative. So long as the multiple-choice 
test and job-performance measures correlate reasonably well, however, the 
change in response topography is an acceptable measurement procedure, 
especially when the multiple-choice test has ben constructed in an ingenious 
way so as to involve more than simple recognition. Correlated-behavior meas- 
ures, such as paper-and-pencil tests, are often useful in assessing those aspects 
of the task which involve "cognitive" skills such as reasoning and decision 
making, even though such tests may not evaluate proficiency at "simpler" but 
important components of over-all performance, such as manual dexterity- 
Thus, it is not surprising that tests measuring troubleshooting logic frequently 
have only a low correlation with actual job performance when proficiency on 
the job includes the location of troubles by simple techniques, such as looking 
for a broken connection or an obviously damaged component. Finally. it 
should be noted that not all tests involving verbal skills are necessarily meas- 
ures of correlated rather than actual job behaviors. Assessing performance 
on a task such as the preparation of reports or records can be achieved with 


paper-and-pencil tests and yet involve little if any remoteness from actual job 
behavior. 


Varieties of Proficiency Measures and Their Relation to Training 


As noted in the beginning of this section, proficiency tests have been 
categorized in a variety of ways. The classification system used in this chapte” 
is based on a continuum of remoteness between the situation represented by 
the proficiency measure and the behaviors elicited by it, and the corresponding 
situation and behaviors appearing in the actual job environment. In some 
instances, it is feasible to assess proficiency on the job; in most cases, how" 
ever, some degree of simulation or change in response topography is neces" 


—Robert Glaser and David J. Klaus - 463 


sary for measurement purposes. The degree of remoteness from the actual 
job situation employed in a proficiency measure is a function of the amount 
of standardization possible in the job situation, the cost of test administra- 
tion, the feasibility of observation, the accessibility of the behavior, and the 
complexity and frequency of the task. 

A final point to be made in considering types of proficiency measures is 
their relationship to stages of training. Early in training, when proficiency 
in large part represents familiarization with the job and with the verbal 
directions and nomenclature used, correlated-behavior measures of proficiency 
are probably the most useful. In support of this notion, research has shown 
that the intellectual and verbal components of a task are generally em- 
Phasized early in training (Fitts, 1961). Only after training has progressed 
does performance become more or less automatic. As the verbal components 
become increasingly covert, they not only are less amenable to measurement, 
but by that time, simulated or on-the-job performance measures are more 
apt to sample adequately the more integrative, complex, and relevant 
aspects of terminal behavior. As a result, simulated and on-the-job meas- 
Ures are frequently not efficient unless there have been substantial amounts 
Of preceding training. In addition, the job situation may involve potential 
damage to equipment and trainees and thus preclude the possibility of 
Proficiency testing on the job until training is relatively complete. In driver- 
training, for example, the goal may be to produce skilled and careful 
automobile drivers, but it would be difficult to assess behind-the-wheel 
behavior until after some of the basic principles and essential skills of 
driving have been learned. It is likely that the continuum along which pro- 
ficiency tests vary from correlated measures through simulated measures to 
On-the-job measures parallels the level and kind of training then in progress. 


SOME APPLICATIONS OF PROFICIENCY MEASUREMENT 


The approaches to proficiency measurement described thus far point to 


a number of the considerations involved in developing or using an instrument 
to assess human performance. In this section, the application of proficiency- 
Measurement techniques to three representative man-machine tasks will be 
described, In each of these areas, some typical performance-evaluation studies 
Will be used to illustrate methods employed for measuring proficiency in exist- 
areas that will be examined reflect a broad picture 
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were evaluated and compared, in order to develop systematic and reliable 
tests with which to assess pilot performance. As described in a comprehensive 
report of these Army Air Forces efforts (Miller, 1947), the approaches could 
be categorized into five principal groupings. These were general subjective 
evaluation, subjectively rated work sample, objective observation, mechanical 
or photographic recording, and printed tests of flying information. 

The use of general subjective evaluations of pilot performance was an 
attempt to evaluate proficiency by means of an over-all rating. Typically. à 
flight-school instructor flew with a student on a training mission and, at its 
completion, assigned the student a grade or rank with respect to his perform- 
ance on the entire mission. Efforts to increase the reliability of these ratings 
by anchoring the high and low ends of the scale with behavioral descriptions 
yielded reliabilities or agreement among the judges which were not uniformly 
high, ranging from .35 to .89 in various studies. In many cases, the level of 
agreement was thought to be inflated as the result of halo effects, due tO 
knowledge of previous evaluations and the classroom performance of trainees. 
When specific aspects or work samples of flying were rated instead of over" 
all performance, the intercorrelations among scores for various maneuvers 
were quite high, further suggesting that there was a rating halo affecting 
judgments of these specific samples due to the instructor’s over-all impression 
of the trainee’s performance. The difficulties encountered in using subjective 
ratings was further indicated by the low correlations obtained between judg- 
ments made at different levels of training. When the student advanced to ? 
new instructor and a new training environment, his proficiency ratings at me 
new level correlated only between .20 and .35 with ratings made at the 
previous level. 

While neither of these two approaches to subjective ratings proved very 
effective, observation techniques which were more objective tended to reduc? 
the amount of variability attributable to the Observer, and hence yielded more 
useful scores. A large number of objective check lists of pilot performance 
were prepared which were designed so as to be applicable both to complet? 
missions and specific maneuvers. The check lists included items representin? 
observations such as cruising speed, throttle setting, and degrees of ys 
which could be scored on the basis of control settings and instrument reading” 
rather than subjective judgments of pilot effectiveness. 8 

Check-list observations were found to yield correlations as high as 8 
between judges observing the same flight, indicating the levels of reliability 
that can be achieved with this technique. However, test-retest reliabilit 
when two sets of observations were made on the same day but not on E 
same flight were no higher than .32. When the observations were made t 
independent observers on different days and with the trainees using differe 
airplanes, the highest reliability coefficient obtained was .04 (Miller, 194 $ 
p. 143). The objective check-list approach, then, does reduce variabili 
attributable to judges, but the variability in over-all performance from day 
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day or even from one part of the day to another suggests that fluctuations in 
weather, condition of the equipment, or condition of the pilot heavily influ- 
ence the scores obtained. As a potential solution, it was suggested that ratings 
be gathered over a long enough period of time to reduce the effect of these 
fluctuations. It was also suggested that reliability could be improved by the 
use of weighted combinations of scores on selected maneuvers which were 
Chosen in such a way as to maximize the stability of the total scores. 

A further effort to reduce the influence of the human observer in pilot 
evaluation was the development of mechanical and photographic instruments 
to record data relevant in making judgments. Devices attached to aircraft 
Controls or displays, for example, can permanently record pertinent pilot 
movements and deviations of instrument needles. One of the best known of 
these devices was the use of motion pictures to record target positions during 
fixed gunnery practice by fighter pilots. Complex recording equipment has 
Not been widely adopted, however, inasmuch as it apparently contributes 
little to the reliability of judgments over and above that which can be ob- 
tained when judgments are guided by means of carefully constructed, objec- 
tive check lists. Smith, Flexman, and Houston (1952), for instance, found 
that Photographically recorded instrument readings correlated between 74 
and .90 with observer recordings. Presumably, however, recording devices 
can add significantly to reliability when the observations to be made are so 
Complex or must be continued over so long a period of time as to make it 
Unlikely that human judges would produce reliable scores. m 

Printed tests of flying information used in the Air Forces si ies ea 
developed to measure kinds of knowledge correlated with flying skill, such as 
Principles of aerodynamics, weather, instrument flying, and jp el 
cedures, As is typical of these kinds of proficiency measures, ] e re - : i 
9f scores on the printed tests were generally high, but the corre eee 
these measures and subjective ratings of flying skill were fairly low. ‘nal 
theless, it was felt these measures did contribute to the curam am = to E 
Performance, and they were adopted to supplement flight checks and ora 
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ponents in the equipment. Because these activities are not scheduled and are 
much less routine than preventive maintenance procedures, observation of 
on-the-job performance generally is not practical. The most elusive aspect 
of corrective maintenance are the skills involved in problem-solving—that is, 
diagnostic troubleshooting. Because of the complexity of troubleshooting 
behaviors and their importance to successful system operation, tests of 
troubleshooting skills have been extensively investigated. 

In a comprehensive survey of troubleshooting measures, Fattu (1956) 
identifies four principal types of proficiency instruments that have been used 
by the military services to assess troubleshooting performance. These are 
on-the-job measures, performance tests, simulator tests, and paper-and-pencil 
or oral interview tests. A brief description of each of these types of tests will 
illustrate the variety of approaches which have been used to evaluate trouble- 
shooting proficiency. 

As already noted, on-the-job assessment of troubleshooting skill is difficult 
because much of the behavior is covert and because the irregularity with which 
equipment failures occur makes it necessary to depend on the judgments an 
ratings of supervisors and peers rather than independent observers. Studies 
have been carried out on on-the-job proficiency in troubleshooting employ- 
ing these measures, such as the one conducted by Demaree et al., (1954. 
1955). They collected supervisor's rankings on four aspects of job perform- 
ance, supervisor's ratings on over-all job performance, and peer ratings °! 
on-the-job proficiency. From their study, they concluded that ratings by pee 
more closely corresponded to estimates of proficiency based on a variety of 
other types of proficiency tests than did supervisor judgments. The over-all 
rating by supervisors was least satisfactory in this respect, because of differ- 
ences between supervisors in their use of the rating scale. 

A typical performance or work-sample measure of troubleshooting PIO" 
ficiency was developed by Rulon and his associates (1954). These invest 
gators modified items of equipment with which trainees were familiar, so that 
the examiner could operate a switch in order to insert preselected malfunc- 
tions into the equipment. The malfunctions used in the test were selecte 
from a large sample of possible troubles. Particular malfunctions were diS- 
carded from the sample if they were too difficult to insert, produced unreliable 
symptoms, or required testing time out of proportion to the information 
obtained. Each troubleshooting problem was divided into steps or segments: 
for convenience in scoring, and to allow the examiner to redirect the student? 
at each step so that students who had made errors would not be over 
penalized in later portions of the same problem. Detailed records of perform 
ance were made during the test, and these observations were later score 
according to the amount of time Spent on a problem and the number of times 
an examinee incorrectly selected defective components. The records obtaine 


were also used to analyze troubleshooting approaches and the techniques that 
were used to locate a malfunction. 
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Unlike the performance measures just described, simulator tests attempt 
to assess troubleshooting skills without using actual equipment. A number of 
different simulators have been developed for this purpose. In general, the 
examinee is provided with a schematic diagram which identifies test points 
Where he might obtain instrument readings or other information if he had 
access to the actual equipment. The student can then use the simulator to 
Obtain whatever information he desires in order to locate the malfunction. 
pus simulator of this type is the Tab Test (Glaser, Damrin, and Gardner, 

4). [n using it, the examinee selects from a list of 30 or 40 checks those 
he might try in the job situation. When he selects a check, the examinee pulls 
Up a tab revealing the information he would obtain had he actually performed 
that check on operating equipment which had the malfunction being simu- 
lated. A mechanical device developed for similar proficiency measurement 
Purposes is the Automasts (Bryan et al., 1954). This device also allows the 


Student to select test points in the order in which he would investigate them, 


and permits him to choose the kind of information he would like about each 
g, or oscilloscope pattern. 


a Point, such as an ohm reading, volt reading 
imulator tests of troubleshooting proficiency can be scored on a variety of 
number of checks to solution, time to solu- 


Performance aspects, such as the 
ber of incorrect attempts at identi- 


oe tedundancy of performance, or num 
ying the defective component. 


Because of the sequential nature of the troubleshooting task and the 


Interdependence of checks, the measurement of troubleshooting performance 
with familiar forms of paper-and-pencil tests imposes a high degree of arti- 
ficiality, It is possible, however, to use multiple-choice and related test items 
to measure behaviors likely to be correlated with troubleshooting skills. 
üüpe (1955), for instance, developed a multiple-choice test of basic elec- 
tronic knowledge consisting of 130 items. The test measured knowledge of 
terminology, procedures, schematic diagram interpretation, application of 
Principles, and analysis of symptoms. f 
The correlations among the various types of troubleshooting measures 
Mentioned in this section are of interest because they illustrate the relation- 


ships between proficiency tests having different degrees of remoteness from 
ave been conducted in the arca of 


pede performance. Several studies h onduc 

ubleshooting performance to determine these relationships. Saupe (1955), 
Or example, obtained a correlation coefficient of .55 when he compared 
Scores on his paper-and-pencil test with a performance test measuring pro- 
ciency at troubleshooting actual equipment. Grings and his associates 
(1953) correlated performance, paper-and-pencil, and simulator measures, 


fc Measuring the troubleshooting skills necessary to perform corrective main- 
Chance on the same item of equipment. They found a correlation coefficient 
and the simulator test, which was 


measure 
Test described above. The paper-and- 


nsisted of troubleshooting items as 


‘56 between the performance 
Pe Carlier version of the Automasts 
"cil troubleshooting measure, which co 


an 
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well as items measuring general electronics knowledge, correlated .64 with 
the performance test and .46 with the simulator test. These investigators also 
obtained data on the relationship between the three tests and supervisor 
ratings of general troubleshooting performance on the job. The ratings by 
supervisors correlated .50 with the performance measure, .21 with the simu- 
lator measure, and .46 with a paper-and-pencil test. A specially modified 
version of the simulator test produced a correlation coefficient of .52 with 
supervisor ratings but did not materially affect the relationship between the 
simulator test and the other two measures. 

Glaser and Phillips (1955) obtained correlation coefficients ranging from 
.74 to .88 between a paper-and-pencil test of troubleshooting proficiency and 
various scores derived from a version of the Tab Test. Scores on their simu- 
lator test correlated .54 to .70 with a paper-and-pencil test of general knowl- 
edge of the equipment. In a comprehensive study comparing various types 
of troubleshooting proficiency measures concerned with the same equipment, 
Demaree and his associates (1955) reported that paper-and-pencil, simulator, 
and performance measures had no strong relationship among themselves. 
They found that of these three measures, only the paper-and-pencil tests 
were very predictive of on-the-job performance, as measured by either super- 
visor rankings or peer ratings. It is difficult to generalize from the results of 
these studies, since it seems likely that the specific characteristics of the 
tests—such as the type of simulation, the form of paper-and-pencil items, the 
nature of the on-the-job measures, and the complexity of the equipment 
involved—all influence the correlation coefficients that were obtained. 


Multiman Systems 


In the complex man-machine systems of today, the human component is 
very likely to consist of a team of individuals. With respect to proficiency 
measurement, less work has been done in this area than has been carried out 
with individual performance, yet the importance of developing proficiency 
measures for multiman systems is being more frequently considered (com 
pare Chapters 11 and 13). Several approaches for assessing group perform" 
ance have been developed, both for small, face-to-face teams and large-scale 
complex organizations, although the number of these studies is quite 5m? 
when compared with the number of projects that have been carried out e 
develop tests to assess the proficiency of individuals. 

A proficiency measure developed to evaluate shipboard Combat Informa" 
tion Center, Gunnery, and General Navy teams is described by Glaset 
Glanzer, and Klaus (1956). Their instrument, the Team Performance Record, 
was designed to be used in diagnosing team performance during training 
exercises through records which identified performances needing improve” 
ment and performances to be encouraged. In using this measure, the ob- 
server's attention was directed to thirteen specific categories which previously 
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had been found to be critical in differentiating high- and low-rated teams. 
The categories under which observers recorded effective and ineffective 
actions included such aspects of group performance as "composition of group 
and assignment of members," “communication procedures and coordination 
of information," and "interchangeability and assistance among team mem- 
bers.” When used over a number of exercises, the Team Performance Record 
provided a series of measures which could be used both to evaluate teams 
and determine areas of weakness where additional training was required. 

Another example of proficiency measurement applied to crew perform- 
ance is described by Cook and Baker (1960). They employed several check 
lists, containing almost 500 specific items, to evaluate the actions of tank 
crew members, tank crews, and tank platoons consisting of five tank crews 
each. The observers watched the performance of the tank crews during 
30-hour field exercises involving several kinds of tactical maneuvers, defen- 
sive and offensive actions, and special procedures. Observations were aided 
by allowing the judges to listen in during radio communications between 
crews. Each exercise was carefully standardized in terms of the terrain, aggres- 
Sor action, and battle objectives assigned to the crews. The check-list items 
Consisted of carefully described events, so that the observer only had to 
determine their occurrence or nonoccurrence and did not have to make elab- 
Orate, subjective judgments. During one exercise, when five observers inde- 
Pendently scored crew performance, there was an average over-all agreement 
of 82 percent among scorers. In addition to the check list, a record was kept 
of the number of targets hit during range firing of live ammunition. 

A different approach was employed by Chapman and Kennedy (1956). 
Their measures, which were developed for use in studying large experimental 
groups in laboratory settings, included scores which measured "effort ex- 
Pended" by the group and the group's ability to differentiate among items of 
information of varying degrees of relevance to the over-all task. "Effort 
expended” was defined in terms of the number of items of information utilized 
by the group in accomplishing a task. The group's ability to differentiate 
information input was defined in terms of how effective the group was in 
Selecting the most relevant information from that made available to it. The 
Tesults of these studies indicated that as the task load of the group was in- 
Creased, "effort expended" did not show à proportionate increase; the group 
Was able to maintain proficiency by more effectively selecting relevant infor- 
mation and by developing better procedures for handling each item of 


Information. 


The systematic measurement of proficiency in very large-scale systems 


isa relatively recent accomplishment. The most extensive of these efforts is 
the work on the North American Air Defense system (NORAD) reported by 
Carter (1961). During one NORAD exercise, exercise "Desk Top," the com- 

ined effectiveness of some 15,000 participants, representing more than 20 
Bovernment agencies and military units, was assessed. The primary purpose 
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tify and describe the components of proficient job performance and for deter- 
mining the relative weighting of these components with respect to a given 
task. Advances must be made in techniques for appropriately sampling job 
performance, for establishing the reliability and validity of proficiency meas- 
ures, and for specifying the kinds of test situations appropriate for eliciting 
various complex behaviors. 

The degree of simulation required to elicit joblike behaviors, for example, 
cannot be determined in any systematic fashion, and there are no principles 
for deciding which features of the environment must be simulated for pro- 
ficiency test purposes. Tests which are easily and inexpensively developed, 
administered, and scored may assess aspects of performance which are only 
remotely similar to on-the-job behaviors. On the other hand, extensive simu- 
lation is often employed, at considerable expense, to insure that the job 
environment has been adequately replicated in the test situation. lt is likely 
that between these extremes some optimal degree of simulation exists which 
permits a valid evaluation of proficiency at reasonable cost. A basic psycho- 
logical problem is the need for more detailed knowledge of the kinds of 
stimuli likely to elicit particular behaviors. 

Further efforts also should be directed at the development of additional 
techniques for assessing the competence of performance. The most commonly 
used measures of proficiency are the degree of accuracy and speed of perform- 
ance, and the occurrence of the desired response in a particular situation. Less 
emphasis has been given to other measures which may provide additional 
information about performance. Some examples of these are scores repre- 
senting the characteristics of errors, the degree of retention of the response. 
the susceptibility of the response to decrement as a function of factors such 
as stress, and the likelihood of appropriate responses transferring to a new 
situation. 

Finally, assessing the proficiency of the human component in a man- 
machine system requires the development of a precise terminology fO" 
describing behavior, in order to specify performance limits, job requirements, 
and existing performance levels. Knowledge of the relationships among these 
three descriptions of performance will facilitate the optimum utilization of the 
human component in system operation. 
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WHEN A SYSTEM AND ITS COMPONENTS — 
equipment and human beings—have been designed and 
developed to the point at which actual operations secm 
both feasible and impending, questions about the em- 
ployment of the system come to be of pressing impor- 
tance. It is at this stage that specific procedures are 
planned and put into effect for the purpose of evalu- : 
ating the system as a whole. Although earlier conceptions of the need om 
“system test” tended to be described in terms of the proving of hardwa 

characteristics, the development of modern systems has made the — 
of such ideas abundantly apparent. System evaluation is a phrase which 
implies the necessity for systematic observation and measurement of a recede 
capabilities in terms of its total functioning, including and even piae 
the integrated operation of its subsystems with their human and equipmen 
components. m "m 

The purposes of system evaluation, as seen in this chapter, are to Ia d 
possible a prediction of the System's capability to perform its major "i 
alternative missions, to compare these performances with those of other Sys" 
tems designed to accomplish a similar mission, and to develop the Hem 
of system usage which will optimize its performance. Such purposes can bes 
be achieved by broad-scale employment of simulation techniques, which 
make it possible to represent the inputs to the system, the conditions of n 
employment, the environment within which it is designed to function, an 
thus to permit the collection of data providing valid conclusions about system 
operations. 

The approach to system evaluation described here has many close resem- 
blances to scientific experimentation. The analysis of problems to be studied, 
the design of techniques of experimental control, the selection of performance 
criteria, and the formulation and testing of specific hypotheses, all display 
formal similarities to those occurring in the more familiar setting of the scien- 
tific laboratory. The design and control of studies for system evaluation pose 


of the entities of study as crews Performing integrated system functions, rathe 


, 
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Although the interest in evaluating and improving systems is rooted in 
the remote past, the application of scientific techniques to this end is of rela- 
tively recent origin. From the beginning of written history, men appear to 
have been interested in assessing the adequacy of systems and improving them 
—particularly political, economic, and other social systems. Until recent years 
Such efforts have been largely speculative and philosophical. But in the past 
few decades the emphasis has shifted from interest to necessity and from 


Speculation to scientific experimentation. . . . 
This shift in emphasis probably began about the time of the industrial 


Tevolution and still continues. There are at least two reasons for relating the 
Origin of the scientific interest in systems to the industrial revolution. First, 
there is an intimate relationship between the application by industry of scien- 
tific discovery and the stimulation of scientific enterprise by industry. The 
industrial and the scientific revolutions developed symbiotically and in parallel. 
Second, the man-machine system, which is in many ways a symbol of the 


* Portions of this chapter were written while the authors were on the staff of the Data 
Systems Laboratory of the Thompson Ramo Wooldridge Corp., Denver, Colo. 
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industrial revolution, is—unlike political and economic systems—amenable to 
study by means of the experimental methods of science. 

The characteristics of the industrial revolution embrace the development 
of the factory system, the application of power to industry, the mechanization 
of both industry and agriculture, and perhaps most important of all from our 
point of view, an amazing increase in the speed of communication and trans- 
portation. Fast, efficient and complex machines have become a part of our 
daily life, and because the size of machines influences their cost. speed of 
movement, and case of handling, a premium has been placed on compressing 
them into smaller and smaller packages. As a result, machines have imposed 
greater and greater demands on the men who use them. Elementary system 
functions, previously performed by unskilled or semi-skilled workers, are now 
casily automated, and the remaining highly specialized functions which man 
performs in systems today frequently tax his physical, perceptual, and intel- 
lectual capacities. 

As a result of this, the demands which today's machines place on the 
men who use them have reduced in many ways “the degrees of freedom" 
available to the system designer. There may, for example, be a dozen or more 
satisfactory ways of displaying visually the speed of a motor vehicle which 
does not exceed 90 mph, but there are only a very few satisfactory ways of 
displaying the speed of an aircraft traveling at 1200 mph. As the demands 
placed upon the human operator of a system increase, there are fewer and 
fewer "right" ways of presenting information to them to utilize their percep- 
tual and motor capabilities. Taylor (1960) summarized these facts with the 
statement that "engineering psychology began with the intellectual discovery 
that man is not a perfectly adaptable organism.” 

But the man-machine system designer of today is faced with other sets of 
problems in addition to those commonly associated with human engineering. 
His systems require so long to develop that there is considerable danger of 
their being outmoded before they become operational. Modern weapon SYS- 
tems provide an excellent example of this. In contrast with the weapons of 
A.D. 1000, which had a useful life of 400 years, today's weapon systems have 
a useful life of only three to seven years and yet require about ten years tO 
develop (Johnson, 1960). Today's systems are, in addition, so complex (and 
expensive) to operate that the designer is unable to use actual trial and erro" 
to solve basic design problems. Even in peacetime the efficient operation of 
modern weapon systems on the first trial may be a matter of life and death, 
and many countries depend upon information Processing systems which, if 
inefficiently operated, could lead to a worldwide disaster. There is, in short 
little room for error in most of today’s Weapon systems, and yet there is "° 
opportunity for an orderly gradual evolution of their capability with a view 
toward improving them before they become operational. 

The trend toward the integration of high-speed digital computers int? 
man-machine systems in the last decade has greatly reinforced the need tO 
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evaluate and optimize systems prior to their utilization in real life. Systems 
which attempt to wed men, computers, and other equipments into an inte- 
grated whole are generally permeated with interactions which can virtually 
not be ferreted out without actually testing and evaluating them in some 
formal fashion. 

Not only are computerized systems more complex in their interactions 
than others, but many of the functions and activities performed by these com- 
puterized systems are not readily accessible to system members or to system 
Observers. Some of the system's activities are, in effect, "invisible" (Kepner 
and Tregoe, 1959) in the sense that the adequacy of a given action is not self- 
apparent to observers outside the system. The 
degree to which a given system activity or function is visible will depend on 
the provisions the system designer has made for displaying it. The activities 
all man-machine systems are spread along a visibility- 


but computerized systems have an extremely large 
and models of their com- 
mely difficult to design, 


evident to system members or 


and functions of 
Invisibility continuum, 
number of invisible functions buried in the symbols 
Puter programs, and consequently they are extre 
evaluate, and optimize. 


This chapter deals with the problem of evaluating and improving com- 


Plex man-machine systems, particularly large-scale computerized data-proc- 
essing systems. Before a system of this type can be evaluated in a controlled 
Manner, the experimenter must have at his disposal a means of creating and 
Manipulating the context within which the system operates. Simulation is a 


Valuable and important tool for this purpose. In the following pages, there- 
fore, we shall focus our attention on the problem of evaluating man-machine 


Systems using simulation as a means of generating controlled system inputs. 


Purposes of System Evaluation 

as their primary purposes: (1) the prediction 
Of a system's ability to accomplish its mission, (2) the comparison of a sys- 
tem with alternative systems, (3) the improvement ol Systems OF (4) some 
Combination of these. Beyond these more immediate purposes, there are 
frequently others, particularly economic and political ones, in which there is 
a need to demonstrate the worth of a system. 


Evaluation studies have 


Comparison of Alternative Systems. Although only one version of a 
System may be developed. more generally there are competitive versions. 
ere are, for example, alternative ways of defending the County SPAIN a 
attack utilizing a centralized data-processing system. One such system uses 
igh-speed digital computers to detect, track. and identify incoming aircraft. 
t has been argued that a man is a very light, economical, easily transported 
Somputer who excels in pattern discrimination and is easily programmed. On 
the other hand, he has only a very limited capacity for the storage and retrieval 
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of data. Since a computer is difficult to program and has almost no capability 
to discriminate patterns, it can be argued by this analogy that such a system 
employs a computer in a less than optimal way; and that men should be used 
for the detection and tracking functions which involve figure-ground discrimi- 
nations, whereas the computer should be used for the storage and correlation 
of flight paths. Clearly, arguments by analogy are not particularly scientific 
or convincing. But the possibilities suggested by this argument show that there 
are alternative systems for dealing with the air-defense problem, some of 
which do not even employ computers. It may be desirable to compare and 
evaluate them in a systematic manner. 

In evaluating alternative systems, it is important to inspect the system 
design criteria very carefully. A system designed to process one type of data 
may show up very poorly when compared with a system designed to process 
another type of data. If tested against subsonic aircraft, one air-defense system 
may prove superior to another, but the previously superior system may fail 
completely when the test employs supersonic aircraft. In other words, there is 
frequently an interaction between criteria and systems, and the effectiveness 


of one system relative to another usually depends upon the evaluation criteria 
selected. 


Improvement. Evaluation is actually the first step toward the optimiza- 
tion of a system. In the course of evaluating a system, the experimenter begins 
to isolate problem areas which when corrected must be followed by another 
evaluation. As a matter of fact, system optimization is largely an iterative 
process of system evaluation —> system modification —> system evaluation 


—> system modification, and so on, until some satisfactory design specifica- 
tions have been met. 


Prediction of System Performance in Use. Systems are sometimes eval- 
uated solely for the purpose of estimating or predicting their effectiveness 
when in operational use. Frequently the cost of system evaluation is so great. 
however, that the evaluation is conducted for all three purposes simulta- 
neously. It is also true that there are man-machine systems so extensive and 
costly that alternative systems will never be built and the only conceivable 


purposes of evaluation are the prediction of effectiveness and system improve- 
ment. 


An Exemplar System 


In order to have available referents to which the discussion of systems 
can be tied, a specific example will be developed in some detail. Air-defense 
direction centers are excellent systems to use for this purpose, because they 
have been studied extensively by psychologists and because their functions 
include many of those performed by other information-processing systems. 
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The mission of an air-defense direction center is to detect, identify, and 
track aircraft within a defined area of responsibility. When a detected aircraft 
cannot be identified with the information available at the direction center, an 
interceptor is assigned against the unknown aircraft for direct visual identi- 
fication by a pilot. If visual inspection indicates that the aircraft is “hostile,” 
it is either forced to land or shot down. 

Direction centers use radar to collect raw data about aircraft in the 
vicinity of the site. This raw data is presented on radarscopes to human 
operators, who detect aircraft returns and track them. The operator "calls" 
tracks (via telephone lines) to men stationed behind a vertical plotting board 
(Figure 13.1). The plotting board is made of edge-lighted plexiglass, so that 
aircraft tracks drawn on it with special crayons will be visible throughout the 
direction center. Men stationed in the direction center use this display to 
perform their various functions. 

One of the most important of these functions is the identification of 
tracks drawn on the vertical board. Flight plans are required of all aircraft 
flying at given speeds and altitudes within specified zones. The identification 


r Defense Direction Center taken from 
f the room. The large vertical, 
with its assocvated "tote" board is visible at 


Figure 13.1. Photograph of an Ai 
an observation dais located in the back o 
edge-lighted plotting board 
the far end of the room. 
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is accomplished by correlating a track's position, its speed, direction of move- 
ment, and calculated altitude with flight plans forwarded to the site by the 
Federal Aviation Agency. When an aircraft cannot be identified, an inter- 
ceptor must be assigned against it from the available inventory of weapons. 
Status information about the inventory of weapons, weather, and the like, is 
displayed throughout the direction center on “tote” boards placed alongside 
the large traffic display board. 

Interceptors are guided to their targets by ground controllers. Conse- 
quently, weapons assignment includes not only the specification of the type 
of interceptor and its point of origin, but also the individual controller respon- 
sible for guiding it by radar to its destination. The various consoles within the 
direction center all have the same basic presentation; the controller uses the 
same radar display as that used by the Scope operator initially detecting 
the aircraft. Since the unknown aircraft is within the zone of responsibility 
of the direction center and since the interceptor will also be within the same 
zone, the two tracks (unknown and interceptor) can be merged. All the tasks 
described require considerable skill, but the merging of the two aircraft is 
perhaps the most difficult of all. If the tracks are to merge, the officer con- 
trolling the interceptor must take dozens of factors into account, such as 
direction of wind, relative speed, and altitude. 

A direction center is imbedded within a larger community called the air 
defense division, which normally consists of about ten individual radar sites. 
Any given radar has a limited range, and therefore many radars are needed 
to defend a particular area. Hundreds of radars are required to defend a large 
land mass such as the North American continent; consequently, air divisions 
must be tied together into still larger units. Furthermore, radars overlap in 
coverage, and it is necessary for these various systems to be interconnected 
with one another and to Pass information back and forth. Communication of 
this kind between sites is called “cross-telling.” The need to do this rapidly 
and accurately greatly complicates the over-all problem of air defense. 


SIMULATION AS A TOOL FOR SYSTEM EVALUATION 


Simulation techniques, conceived in their broadest sense, have been used 
by psychologists for many years. As the term simulation is used in this chap- 
ter, it means reproducing or representing a specific set of environmental 
conditions. The difference between the use of simulation in the standard 
psychological experiment and in what is here called the “system evaluation 
experiment” is largely a matter of degree. In the standard psychological 
experiment, the experimenter abstracts and simplifies large chunks of the 
environment; in the system evaluation experiment, he attempts to include to 
a greater degree the complexity and richness of the environment. Experiments 
in these two areas tend to be distributed along a continuum of abstraction- 
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Many psychological experiments, such as those involving a Skinner box. 
T-Maze, or memory drum, have very little direct relationship to the natural 
habitat or environmental conditions of the animals being studied. Such experi- 
ments tend to be at one end of the continuum. At the other extreme are such 
Studies as those reported by Chapman and his associates (1959). During these 
latter studies all relevant aspects of an air-defense direction center and its 
environment were simulated, and the behavior of four air-defense crews 
studied in relation to a variety of input conditions. In between these two 
extremes of abstraction and concreteness, there are numerous psychological 
experiments from various specific disciplines involving different degrees of 
€nvironmental simulation. 

Although simulation is not essential to the scientific evaluation of sys- 
tems, it is widely employed for this purpose, and when properly used, it is an 
Extremely powerful tool. Simulation is used in the evaluation of systems in two 
quite different ways. Sometimes it provides only the inputs to an operational 
System or system mock-up, in which case the responses of the system to these 
Simulated inputs are studied and analyzed. Simulation is also arranged to 
Tepresent the internal functioning of the system itself and not merely system 
inputs, When this technique is used for system evaluation, the entire process 
I5 commonly accomplished on a high-speed digital computer. In recent years, 
a number of psychological studies have been reported in which the internal 
functioning of the organism as well as the inputs was simulated (Ashby, 
1952; Newell, Shaw, and Simon, 1958; Rosenblatt, 1958; Hovland, 1960). 


Simulating System Inputs 


When simulation is used to generate system inputs, the integrity of the 
: yned. Insofar as possible, the men 


Organizati ' men and hines is mainta 
"ation of men and machine i 3 i A 
and the machines are kept as they are 1n real life; although the inputs may 


€ simulated in a laboratory setting or in the field, the illusion of reality a 
Maintained. Simulation is used in this way to study a system in san repe 
Configuration, and the simulated inputs are. of course, ins d y in the 
Sense that they have been deliberately prepared by ett ak i 

A distinction should be drawn between the terms “real . an e -i E 
they apply to evaluation experiments. Realism is a characteristic ihe ate 
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have very little to do with realistic preparation for the ultimate mission of the 
group, and simulation may be the only way to achieve realism. 

The conventional psychological approach to experimentation is one of 
abstraction. The intent of the psychologist is to construct a highly distilled 
experimental situation which will permit the study of selected independent 
variables. The richness and variety of the real-life environment is deliberately 
omitted in favor of the crcation of a completely controlled habitat; the experi- 
mental situation is chosen with care, to guarantee that subjects will have very 
little freedom of response selection; and subjects are selected for the particular 
skills which they bring (or do not bring) to the experimental situation. 

The most important characteristic of the conventional experimental 
approach is that it provides the scientist with an abstract environment—that 
is, isolated independent variables can be studied carefully, their relationship 
to behavior analyzed, and their interactions determined. Although the experi- 
mental psychologist is concerned with the realities of the subject's environ- 
ment in the selection of independent variables and the performance measures 
to be taken, he has very little concern with replicating precisely the typical 
behavioral environment of subjects. Unfortunately, the frequency of inter- 
actions between psychological variables is so great as to place a limit of use- 
fulness on this process of. isolating independent variables for study. Indced, in 
psychological experimentation, it is frequently the interactions which are of 
greatest interest. 

In the simulation approach to the evaluation experiment, the emphasis is 
placed on providing subjects with a more complete experimental situation— 
that is, one closely resembling the real-life environment of the subject. The 
intent of the experimenter is to construct an experimental situation which 
corresponds as closely to the normal operating environment as possible. TO 
this end, subjects are placed in an environment of machines which duplicate; 
wherever possible, the operational situation; insofar as it is practical, the 
properties of the inputs to the experimental situation duplicate those of the 
operational situation; subjects are chosen from the same population as opera- 
tors in the operational situation, and the culture of the population is preserved. 

Three of the consequences of these efforts are similar to those achieved 
using the abstracted experimental situation: (1) although the variety of 
responses possible is much greater, experimental subjects still respond in à 
clearly defined and specified manner, and thus one may plan for and collect 
specific data on their behavior; (2) controls can be implemented and experi- 
mental manipulations arranged, to guarantee the operational validity of the 
independent variables chosen for study—that is, to insure that causal rela- 
tionships can be determined; and (3) subjects are chosen for those skills 
which they bring (or do not bring) to the experiment. Thus, the main differ- 
ence between the conventional approach to Psychological experimentation and 
the simulation approach to system evaluation is the emphasis which the latter 
places on the preservation of a more complete experimental environment, aS 
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Opposed to a more abstracted experimental environment, and the consequent 
Possibility of making highly specific predictions about a particular system, 
rather than the more generalized predictions of conventional psychology. 
(For a more extended discussion of these points, see Chapter 14.) 

There are a number of advantages in the use of simulated inputs to 
evaluate a system, as opposed to the use of so-called live inputs. First, simu- 
lated inputs are precisely controllable. They permit the experimenter to main- 
tain what has sometimes been called "stimulus control." If, for example, one 
is interested in evaluating the ability of a new data-processing system to route 
and control civil air traffic in the United States, it is conceivable that “live” 
inputs or actual aircraft could be used to test the system. The probability of 
Precisely controlling live inputs in order to provide a base against which to 
evaluate system performance is, however, quite remote. Second, simulated 
Inputs permit the experimenter to replicate precisely his stimulus conditions 
80 as to draw accurate comparisons between alternative system configurations. 
Third, although simulation is often quite expensive, other things being equal 
It is generally more economical than using live inputs. Fourth, once techniques 
for simulating system inputs have been developed, they are available on 
demand, Unlike live inputs, they are not subject to extensive coordination and 
aries of politics or the weather. Although simulation 
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Some Considerations in the Development of a Simulation Vehicle 


One of the most important advantages of simulation is that it allows the 
experimenter to maintain "stimulus control." The term stimulus control means 
that the simulation vehicle is designed to insure the experimenter knowledge 
of—and control over—all information entering the system. 

In many respects, the development of a simulation vehicle adequate for 
the purpose of system evaluation can be the most complex aspect of the 
entire evaluation task. For this reason, some of the considerations entering 
into the development of the simulation program deserve special consideration. 
One of the most important of these considerations is the isolation of relevant 
inputs into the system and the decision as to which of these inputs will be 
simulated, that is, the extent to which realism will be preserved. 

The psychologist confronted with the task of creating simulated inputs 
to preserve "control of the stimulus" must decide which of the many inputs 
to a system he wishes to simulate and the extent to which he can deviate from 
"real life" without compromising the kind of experimental predictions he 
wishes to make. Obviously, this kind of problem cannot be solved solely on 
the basis of experimental considerations. Economic and other factors, par- 
ticularly time, force the psychologist to compromise, but he must understand 
the system and his objectives if he is to decide simulation questions intelli- 
gently. 

Let us consider an air-defense system. Radar impulses are propagated 
into the space surrounding a given site. Certain of these impulses strike an 
aircraft or other flying object and are reflected back to the receiver of the 
radar set. These appear on the radarscope as small blips of light. These blips 
constitute the information to be processed, and the entire system is designed 
for this purpose. However, other influences will also be reflected in the visual 
display on the radarscope face. Some of these influences have been predicted, 
and the system equipments are designed to deal with them. For example. 
there is a certain amount of residual energy in the receiver itself, which 
results in a random visual effect on the scope face (noise). Radar equipments, 
however, are designed in such a way that the effects of the energy reflected 
from aircraft are augmented so as to minimize the random noise in the system. 
But how much, if any, of this noise should be simulated? 

In addition to system inputs appearing on the face of a radarscope, 
numerous other sources of information can be isolated. Flight plans from civil 
and military agencies flow into the direction center; other radar sites “cross- 
tell" information about radar tracks and flight plans; pilots of interceptors 
talk to aircraft controllers in the radar site; weather reports arrive from divi- 
sion headquarters; and the like. To what extent can the simulation program 
deviate from, or omit, these? 

Besides planned inputs to the system, others may be unwanted. These 
other kinds of information are often called noise. The manner of dealing with 
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these sources of noise will depend on the needs of the system for the kind of 
information represented by the noise. For example, clouds and heavy weather 
are reflected as areas of intense textured brightness on a radarscope. These 
displays can be valuable adjuncts to weather prediction, and in such cases, 
the system may be modified so that certain radar consoles display this infor- 
mation for purposes of weather prediction. From the point of view of an 
operator controlling aircraft, the weather on a scope may represent "noise," 
whereas from the point of view of a weatherman it will represent information. 

Which of these many inputs coming into a system should be simulated? 
Generally speaking, when used for evaluation purposes, the simulation vehicle 
must be designed so as to consider all relevant inputs into the system, and 
equally important, it must also consider all the demands for outputs. If the 
Purpose of simulation is purely the evaluation of a given set of components 
and subsystems, the outputs to be considered may be relatively constricted, 
and the boundaries of the system to be considered may likewise be narrowed 
to those information-processing tasks to be considered. If, on the other hand, 
ation is to determine the nature of the contribu- 
ider information-processing com- 
d to include additional 


One of the purposes of simul 
tions which the system might make to the w 
munity, the boundaries of the system must be enlarge 


Portions of this community. 
Once the decision to enlarge the system has been made, these additional 


Portions of the information-processing community may be dealt with in one 
Of two ways. In the first case, they may actually be included in the informa- 
tion-processing system being studied, Under these circumstances, all their 
Inputs, tasks, and functions must be accounted for in the simulation vehicle. 
This course was taken by the System Development Corporation in the prepa- 
ration of an adequate training program for the air-defense system (Goodwin, 
1957), The training program as it was originally developed included only 
Units as large as an air defense division, consisting of an average of about 
10 radar sites. However, it was soon determined that the training exercises 
Should encompass larger command units than the division, and the decision 
Was made to proceed in this direction. A much more extensive simulation 
Program was developed. and this ultimately permitted exercises which simul- 
taneously included all radar sites in the United States. 

A second manner of dealing with the wider information-processing com- 
munity is to simulate the activities of those portions of the larger community 
Which contribute to the inputs, or utilize the outputs, of the system being 
Studied, In this approach, those agencies adjacent to the system in question 
do Not actually participate as subjects in the exercise or test, but their activi- 
lies ang inputs are simulated by members of the experimental (training) team 
Using scripts or other aids. The generic term "imbedding organizations" is 
Sed to describe agencies surroun 


ding the system under study and simulated 
y the experimental team. i "-— A 
Given the air-defense system which has been described and the consid- 
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erations just discussed, how do these get combined to form a total experi- 
mental environment in which controlled tests can be conducted? First, let 
us consider the kinds of simulated inputs required to test experimental 
hypotheses about such a system. As a minimum, these inputs in the case of 
the air-defense direction center would probably include: (1) a means of 
generating the radar displays—that is, moving targets on the scopes; (2) à 
sct of flight plans in an appropriate format to be correlated with the display; 
(3) status information for the tote boards, describing the available weapons. 
weather, personnel, and the like; (4) information about traffic being handled 
by adjacent direction centers; and (5) normal information from higher 
echelons of command. 

If it is assumed that organizations external to the direction center are 
simulated by the experimental team (imbedded), the entire experimental 
environment can be illustrated as in Figure 13.2. The air-defense direction 
center (the system being studied) is shown in white. The experimental team 
shown in the light-gray area (right and left of center) generates controlled 
inputs into the direction center, observes the crew in action, collects per- 
formance data, and simulates the activities and inputs of the various “real- 
life” organizations shown in the adjacent dark-gray areas. 


THE SCIENTIFIC APPROACH TO SYSTEMS RESEARCH 


Faced with the problem of evaluating a large-scale system, a psychologist 
typically turns to his background and training in search of a solution. While 
this is undoubtedly a desirable course of action, there are two important points 
to keep in mind in this connection. First, the area of systems research is rela- 
tively new and consequently has a number of special problems which are 
peculiar to it and which must be thoroughly understood. Second, while the 
scientist’s background and training often provides him with valuable insights, 
they can also blind him to possible solutions. In this c 
experimental psychologists are sometimes 
designed experiments are per se a p 


onnection, inexperienced 
inclined to believe that well- 
anacea; and when confronted with evalu- 
ation problems such as those under discussion, the researchers’ first inclina- 
tion is to devise a controlled experiment. 

Unfortunately, carefully controlled experiments to evaluate large-scale 
systems are extremely costly and require essential ingredients such as methods 
for generating controlled inputs, meaningful hypotheses about the system 
being studied, methods of monitoring the system's performance, and defini- 
tions of the boundaries of the system. In many cases the system being studied 
has never been "turned over" in an operational sense, and it is little short of 
ridiculous to attempt to generate controlled experiments without developing 
a program of research. It is necessary to begin by recognizing that all scien- 
tific experimentation proceeds in steps and that these steps are essential to 
the conduct of controlled experiments. 
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In this section we shall discuss some of the major steps in the conduct of 
a meaningful scientific experiment, and their importance to system rescarch 
in general and system evaluation in particular. First, it is necessary to arrive 
at some clearcut understanding of the phenomena (system) to be investigated. 
This understanding will include a definition of problem areas and the develop- 
ment of testable experimental hypotheses. Second, a methodology or meth- 
odologies must be developed for experimentally investigating proposed 
hypotheses. Finally, experiments and studies must be designed and conducted 
to test hypotheses. The evaluation of a complex system involves welding these 
three elements into a total evaluation strategy or program. Only the first two 
of these three aspects of scientific experimentation will be discussed in the 
present section; problems associated with the design and conduct of the system 
evaluation experiment will be taken up in the next section. 


Understanding the System 


In all scientific experimentation, the investigator's first task is to acquire 
some familiarity with the literature in the area he proposes to study. Onc of 
the most important reasons for doing so is to isolate problem areas and to 
begin to construct explanations for data in the area under investigation. The 
scientist may wish to describe a phenomenon, he may wish to explain it, OT 
both. In any case, he begins by collecting facts about it and by secking to 
arrive at as comprehensive an understanding of it as possible. Scientists have, 
of course, been working in this manner for many years, and the scientific 
neophyte seldom appreciates fully the immense supporting structure provided 
to him by his professional community. In conducting a scientific experiment, 
the scientist seldom starts at the very beginning; usually he has available to 
him scores of published reports in his area of specialization. Perhaps equally 
important is the fact that these reports have been abstracted and indexed for 
easy review and access. Also, in most cases the research methodology in the 
particular arca of interest is highly developed: special experimental apparatus 
has been designed; performance measures and performance criteria have bcen 
specified. Indeed, the research tools available for use by the basic scientist 
are as extensive as the range and development of his science. 

Few of these methodologies are ever directly available to the system 
scientist about to evaluate a large-scale system. Nevertheless, and despite these 
difficulties, the basic approach is the same in both conventional and system 
research. The system psychologist begins by obtaining as complete a descrip- 
tion of the system as possible. In large-scale Systems, obtaining such a descrip- 
tion is usually a complex task, and in point of fact, the truly complete 
description often flows from the evaluation experiments. 

Because complex systems are often divided arbitrarily into personnel 
subsystems, communication subsystems, equipment subsystems, and the like. 
total system descriptions are generally not obtained by inspecting any single 
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ode mee Tah of these so-called subsystems has associated 
: s specifications. The Qualitative Personnel Requirements Infor- 
mation (QPRI), for example, is a required Air Force document which 
describes in detail the personnel subsystem. Most large-scale systems require 
similar documents defining system tasks, the specialized training most appro- 
priate to the accomplishment of these tasks, and the organization of the group 
manning the system. In addition, procedures for the operation of each item 
of equipment in the system are generally developed and documented sepa- 
rately. In the case of computerized systems, computer programs must be 
Written. Computerized systems also often require the development of special 
languages by which the man communicates with the computer and vice versa. 

If the system has actually operated prior to experimentation, link analy- 
Ses may have been conducted to study the communication channels and 
establish the frequency with which they are used. If information-flow diagrams 
have been developed and information-delay times established, these can be 
extremely valuable aids in the understanding of system operation. Moreover, 
developing information-flow diagrams is an excellent way of coming to under- 
Stand a system. Variants of a number of established methods can be used to 
accomplish link analyses and information- and process-flow diagrams. Numer- 
ain discussions of these techniques (Chapanis, 1959; 
an, 1949). From such sources as these, the psy- 
iption of the system which is adequate 
for the determination of problem areas and the definition of experimental 
hypotheses, To a large extent, information of this sort can be obtained from 
Other professional people studying the system. With respect to one aspect of 
this problem, however, the system psychologist must depend largely on his 
Own ingenuity and understanding of the system. This aspect is concerned 
With the establishment of the bounds of the system and its subdivisions for 
the purposes of experimentation—that is, the levels at which evaluation 


*Xperiments will be conducted. 
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Context by a consideration of the inputs for processing and the tasks to be 


accomplished. A component of the system is not necessarily an individual 
Piece of equipment, or even a single piece of equipment and the man who 
OPerates that equipment. A component may be conceived as the complex of 
Machines required by a man to utilize a given input in the preparation of a 
*Pecific output through the performance of a given task. The preparation of 

© outputs, through the use of the machire complex and the given input, 
Constitutes the performance of the task. A very simple example of such a 
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component is the complex of a telephone switchboard and its 2 i 
input to the situation is a vocal request to speak to some siege Hs 
operational procedures involve establishing the correct line and flipping’ a 
switch to ring the individual called. The output is the completed connection 
or the discovery that there is no answer. 

A subsystem is either (1) a sequence of components such that the out- 
put of one component serves as the input for a second, or (2) a parallel 
sequence of components such that each contributes to a common output. a 
the sense that subsystems integrate component outputs, they perform a higher 
order of unifying function than components. On the other hand, subsystems 
perform a subordinate system function, in that their outputs are integrated by 
the system. 

An example of the first of these two types of subsystems would be the 
sequence of components in the air-defense system transmitting information 
about aircraft from a radar site to a higher echelon of control. The - 
component in the series is the radar operator who reads the positions O 
aircraft on a radarscope and calls these over the telephone line to the plotter. 
A. third individual at the site reads the plotted positions of the aircraft and 
cross-tells these to a plotter at the next echelon of command, often many 
hundreds of miles away. The plotter at the higher echelon prepares a second 
display of the positions of the aircraft. 

An example of the second kind of subsystem is the identification sub- 
system of the air-defense direction center. Two different types of inputs me 
involved in the identification function. The first is the display of aircraft tracks 
on the plexiglass plotting board described above. The second is the flight plan 
to the air-movements identification section. of the radar site. These two 
parallel inputs are correlated, and the output is an identification of the track 
as either being “known,” "unknown," or "hostile." 

The reader, by now, must be aware that either of the so-called subsys- 


tems used as examples could also be called Systems. The distinction is made 
between a subsystem and a Syste 


components is called a subsystem 


this larger system. 


The value of a distinction between these three levels for system study 
is twofold. First, the excellence of performance at any level is a function of 
the excellence of performance at the next lower level. Thus, if the component 
tasks are not adequately performed, the Subsystem tasks will not be adequately 
performed, and so on. Second, because of the large number of potential meas- 
ures of performance available and the time required to analyze large masses 
of data, a selection of performance measures must be made, and some criteria 
for selection must be decided unon. As research is done with one level of the 
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System, potential areas of difficulty at the next higher level are discovered, 
and thus some preliminary hints are obtained of what performance measure 
Will be required at the next higher level. Since the performance of the com- 
ponents often changes in the context of the subsystem and system, however, 
this procedure does not eliminate the necessity for designing special measures 
to be used at the component level at the time the subsystem or system is being 
studied. 


Performance Measures and Criteria 


When a large-scale man-machine information-processing system is func- 
tioning, its behavior is extremely complex. The untutored observer, watching 
a well-trained system of this type in operation, seldom realizes that he is 
Observing a highly coordinated and smoothly functioning team. Instead, the 
activities of the system seem confused and uncoordinated. Consider a stock 
exchange: men answer telephones, move from one desk to another, tote 
information on vertical boards visible to other system members, shout to one 
another over the general noise level of the system, and observe a variety of 
displays on which coded information appears. The completely naive observer 
Of such a system is in the position of a spectator at a football game who knows 
Nothing of the intricate planning and rules governing the performance of the 
Players. In some ways, the hypothetical spectator of the football game has the 
advantage; a football stadium is designed to allow the observer to see all 
the important action, and the rules of the game effectively circumscribe the 
Play to a limited field of purview. Nevertheless, unless the hypothetical spec- 
tator has had some assistance, he will see little more than one group of men 
Attempting to ram a ball through to the goal of another. In reality, of course, 
Virtually every move of every player is planned before the ball is set in motion, 
and an intricate set of rules governs the entire game from beginning to end. 

he trained observer, who knows precisely what to look for, can quickly 
evaluate the performance of both sides, isolate points of strength and weak- 
Ness, account for the behavior of umpires as well as players, and even predict 


Uture per etos xtent. 

o E be studied intelligently. the sagaciter must 
know "what to look for." He must know what aspects of the system's behavior 
àre important to the kinds of predictions he wants to make. There are those 
Who will contend that all behavior is important, and it is difficult to disagree 
With this assertion when one realizes the importance of subtle cues for pre- 

iting a response. On the other hand, in the interest of economy, if for no 
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sponses are usually selected on an entirely different basis. The experimenter 
has reason to believe that the responses he has decided to measure are cor- 
related in some way with the variables he is manipulating, and that informa- 
tion about this relationship will enable him to make intelligent and useful 
predictions about the system's future performance. During the course of the 
experiment, he may observe instances of behavior which he believes to be 
meaningfully related to the phenomena under study, and he may, as a result 
of such observations, expand the measures he takes in future experimentation. 

One of the key problems in system evaluation is the determination of 
performance criteria. Only after performance criteria have been established 
is it possible to select intelligent and useful performance measures. Perform- 
ance measures are the standards against which the performance of a system 
is evaluated. In some cases, these standards are defined points along scales 
of measurement. In other cases, they are implicit in the statement of per- 
formance criteria. Performance criteria for a single system may differ greatly, 
depending on the evaluator and his objectives; and because one performance 
criterion frequently conflicts with another, it is always necessary to select 
and choose from those available. An example may help to clarify these points. 

Let us assume that one is interested in evaluating this year’s automobiles. 
Before such a task can be undertaken, it will be necessary to define for what, 
and perhaps for whom, the evaluation is being undertaken. If the automobiles 
are being evaluated for the commissioner of roads, or for the smog-control 
district, the criterion of performance may be quite different from that accept- 
able to a potential buyer. The commissioner of roads may be interested in the 
weight of the car and the damage which it is apt to produce on roads. The 
smog-control director may be interested in the efficiency of the engine and 
the quantity of unburned gases which the car releases into the atmosphere- 
A potential buyer may evaluate the car against any one or a combination of 
criteria, including: (1) economy of operation; (2) case of handling; (3) rid- 
ing comfort; (4) seating capacity; (5) luggage space; (6) cost of mainte- 
nance; and (7) style. Furthermore, individual buyers will differ greatly in 
particular requirements. A salesman, traveling F 
very little about economy of operation but will instead want comfort, ease Of 
handling, luggage space, and style. If his company is buying the car, on the 


other hand, it may be quite concerned with economy of operation and cost 
of maintenance. 


at company expense, may care 


The number of possible criteria against which a system can be cvaluated 
is usually very large, and the evaluator must always select from the ones 
available. Such a selection can only be made when the user's requirements 
are fully understood. This understanding should include not only the impo" 
tant criteria and their relative weightings, but a clear notion as to which of 
the criteria can be “traded off” for others. The need to “trade off” arises 


because individual criteria often conflict with one another. It may not be 


possible, for example, to get economy of operation and riding comfort and 
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luggage space. One may have to be traded off for the other. Similarly, in 
information-processing systems, time and quality may be seen to come into 
conflict. It may be extremely important to process given items of information 
in the shortest possible time, but it is usually necessary to sacrifice some 
quality in order to do so. 

Generally speaking, the measures taken of system performance involve 
some clear-cut base against which to evaluate performance. A criterion fre- 
quently used is perfect performance. Perfect performance may have some 
absolute meaning, or it may have a relative meaning. One measure of system 
performance might, for example, be the time required to accomplish a par- 
ticular task, such as the detection of a penetrating aircraft. Perfect perform- 
ance in the absolute sense might in this case mean the detection of aircraft 
With zero delay or the detection of all penetrating aircraft. Let us cite another 
example, contained in a system designed to process photographic intelligence 
extracted from photographs. If the pictures were first viewed by the experi- 
menter under extremely favorable conditions, with images enhanced and with 
improved ground resolution, a relative basis against which to evaluate future 
Performance of the system could be established. Measures such as these— 
Cither absolute or relative—involve calculating the discrepancy between a 
Criterion measure and the performance of a crew. . . 

Once the question of performance criteria has been settled, it is possible 
to decide upon the performance measures which will be taken to evaluate the 
System against the criteria. These measures will vary greatly from one system 


lo another. In the case of our example of the automobile-evaluation study, the 


Performance measures may include miles per gallon of gasoline, answers to 
Questionnaires, cubic feet of luggage space available, cost of repairs per mile 
driven, and the like. Performance measures employed in information-proc- 


essing systems are commonly latency, error rates, and information-flow rates. 


The Development of Monitoring Techniques 
performance one wishes to collect is only one 


aspect of the task: actually getting them is another. The development of aids 
and techniques for monitoring system performance is a major task in itself. 
y of the pencil-and-paper variety, although 


Aids for this purpose are generall Á 
IN some cases it is most economical to produce them by using a computer. 
t is always necessary to plan their production along with simulated inputs. 
» for example, a simulation program has been developed for presenting to 
©Perators information on radarscopes, and if performance 1s to be evaluated 
d in the previous section, it will be 


Usin exes ie i describe 
€ the kinds of techniques de | aids which, in effect, provide the 


neces ial experimenta 
Ssary to develop special experimen à ^e 
"Perimenters with a running description of what is actually appearing on 


© Scopes, 
An experimental aid of 


Establishing the measures of 


this kind is shown in Figure 13.3. This particular 
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SITE 076. 
TIME 1635 


2442109 
\ 2442099 


= JWADF 27TH AIR DIVISION (DEF: an 
LAMBERT CONFORMAL CONIC PROJECTION IDEFENSE) P-76 
SCALE 


1: 3,440,000 STP FORM 076-03111/561 


Figure 13.3. An example of a computer-generated aid used to follow 
the course of a problem presented to air defense crews. 


aid is called a radar map. It is used in conjunction with the System Training 
Program (Goodwin, 1957)—a program developed to train air-defense com- 
mand crews—and is used by psychologists monitoring the system's perform- 
ance. This example indicates all tracks appearing on the scope for the pat 
ticular sweep of the radar’s antenna occurring at 1635 (4:35 p.m.). In 
addition, certain amplifying information about each of these tracks is printed 
on the map itself and in the margin. Each arrow, for example, represents 3 
particular aircraft position and its direction of flight. Numerical information 
on each side of the arrows is codified to indicate altitude, aircraft seria 
number, and other information about the flight. Using such aids as the rada" 
map, the system psychologist can compare and evaluate crew performance 
against the controlled stimulus input. Such aids as these are, in effect 
“ponies,” and they are essential tools for the study of systems. 
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In addition to such special aids as these, decisions have to be made with 
Tespect to the amount of data to be collected and the way in which it will be 
collected. There is a good deal of difference of opinion as to the amount of 
data which ought to be collected during system runs. Some system psycholo- 
gists believe that good experimental practice dictates the advance specifica- 
tion of an experimental design, the performance measures to be taken, and 
the statistical tests to be applied. At the other extreme, there are those who 
believe that every possible scrap of data should be collected and subsequently 
culled for relevance. Psychologists at both extremes justify their respective 
Positions on the grounds of economy. 

In an absolute sense, it is, of course, extremely expensive to collect all 
Possible performance data during system runs, but in a relative sense, this is 
Not necessarily true. There is very little chance of repeating a system run 
because of the costs involved—and those who advocate collecting all pos- 
Sible data during a run view the relatively small added expense as a kind of 
insurance. There is no one answer to this question. Early in the study of a 
Particular system, arguments favoring the collection of vast amounts of data 
are somewhat more potent than later in the history of the system, when its 
Operation is relatively well understood and the chances of an experimenter 
Oversight are remote. 

The number and type of component or system measures selected will 
ülso depend upon one's knowledge of the manner in which a given inde- 
Pendent variable influences performance, and upon the purpose of the experi- 
Ment—for example, whether one is interested in the system per se or in the 
effects of particular independent variables. If the interest is in subsystems of a 
System, or the system as a whole, the practice is usually to collect as many 
Measures as can be shown to vary with the independent variables used. This 
is à. wasteful procedure, but not an entirely unreasonable one. Systems re- 
Search is a new area of research, and the development of research procedures 
including valid and reliable measures of performances has only just begun. 
The constraints to reduce the number of measures taken are in terms of the 
time available for preparing reports of research and the amount of knowledge 
available about the system. J ! 

If the effect of one or more independent variables on a fixed system is 
the Object of the study, a knowledge of the functions of the subsystems and 


interrelations between subsystems and of the points of effect of the independ- 


"Dt variables in question will allow the experimenter to make a selection of 


Measures which will reflect the nature of the relationship between the inde- 


Pendent variables in question and the performance of the system. It should be 
that this latter type of research—study of the effects 
s—assumes a good deal of knowledge about the sys- 
arch usually comes at a some- 


bur ace a, 
or Vious from the above 
fo Independent variable. 

M in question, In addition, this type of rese 


at later stage in the evaluation of a system- : : 
of techniques for collecting experimental data 


There is a wide range OF c 
bservation is one of the commonest and 


a E 
bout à system's performance. Direct o 
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most widely used methods, and in recent years closed-circuit TV has extended 
the visual range of the experimenter to formerly inaccessible areas. Various 
techniques have also been developed for the remote monitoring and recording 
of auditory information, and these are, of course, also widely used. Whether 
one uses visual or auditory monitoring techniques, closed-circuit TV, or 
magnetic recordings, one of the principal problems is to keep from contami- 
nating the experimental environment. The methodological problems in this 
area are virtually identical with those of the conventional psychological experi- 
ment, and consequently will not be elaborated. 


Experimental Hypotheses about Systems 


It is generally agreed that hypotheses are propositions about the world, 
stated in the form of empirically testable predictions. There are two key words 
or phrases in this definition. The first of these is the word “proposition.” A 
proposition is something set forth or proposed. A hypothesis is a proposed 
solution to a problem or explanation for a fact; it is not formulated as a prob- 
lem or question, but as a solution or explanation. Although hypotheses stem 
from an understanding of the system and a recognition of problem areas, 
they are not statements of problems. The second key phrase in our definition 
of the word hypothesis is the phrase “empirically testable.” By this we mean 
the hypothesis must be stated in such a way that it can be subjected to 
empirical, experimental test, and either accepted or refuted. For this to be 
possible, terms in hypotheses must be defined operationally, and the hypoth- 
eses must be stated in such a way that they can be refuted—that is, possible 
outcomes of the experiment must permit the refutation as well as the accep- 
tance of the hypothesis. The hypothesis is stated, therefore, as a prediction 
about the world, having possible outcomes in support of or in refutation of it. 

We have stated that hypotheses spring from problem areas. What are 
some examples of these problem areas in system research, and what kinds of 
hypotheses result? System problems tend to arise in five or six major areas, 
having to do with the equipment of the system, the personnel manning the 
system, the nature of the inputs to the system, the environment within which 
the system is imbedded, the operating procedures defining how the system 
will be run, and, if a computer is part of the system, the computer programs. 

There are two types of variables with which an experimenter is CON- 
cerned: independent and dependent variables. The problem areas just de- 
scribed represent broad classes of independent variables which may be 
manipulated for the purpose of system evaluation, and each of these classes 
can be broken down into subclasses (Figure 13.4), 

Experimental problems, for example, may concern the allocation of 
functions to men and machines, the format of displays to be used in the sys 
tem, the number of items of equipment required, or the optimum arrangement 
of men and machines. These problems as a class are sometimes thought of as 
human-engineering problems. In the case of the air-defense direction center» 
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Independent variables Dependent variables 
System 
Equipment 1. Configuration 
o 2 2. Human engineering 
EG 
| Personnel » 1. Abilities 
Inputs ES 2. Training 
pu M5 3. Organization Outputs 
l. Amount/Time I 4. Motivation and morale 1. Response 
2. Type - latency 
3. Priority Computer programs 1. Type 2. Error rate 
4. Distribution 2. Number 
5. Noise CHAS 3. Format 
Operating procedures 1. Task assignments 
2, Personnel rotation 
EEN 3. Filtering 
| 
Total environment 
1. Distractions 
2. Command support 
Figure 13.4. The experimental environment for the general case, 


illustrating typical areas within which evaluation hypotheses are generated. 


s area might include the following: men 


Note that the vertical plotting board is found to be too small, and tracks drawn 
On it tend to become confused during heavy raids; a change of format for 
the “tote” boards is suggested; a new item of equipment is proposed, to help 

uty directors vector interceptor aircraft to their targets; and it is suggested 
that covering the scope face with transparent plastic of an amber hue will 


Improve performance. How these suggestions are evaluated will depend on 
many things, not the least of which is money, and it should be clearly under- 
quipment problem areas, some of 


Stood that they are merely illustrations of e 
g, tested in a subsystem or system 


Which might be tested in a component setting, test | : 
context, or perhaps not formally tested at all. Similar kinds of examples could 


€ cited for cach of the other classes of variables of Figure 13.4, but for the 
Purpose of the present discussion it will be more profitable to trace a single 
air-defense problem through a complex experimental evaluation. 


Specific examples of problems in thi 


An Example of an Evaluation Problem | 
the area of operating procedures. In real-life 


Our example comes from 
lar crew change from week to week and even 


9 1 H 
Perations, the men of a particu 
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from day to day. For a variety of reasons, different men work at the same 
position within a given crew. Often these changes in crew composition are 
temporary and occur because of illness or because other duties interfere. 
Sometimes the changes are more permanent and occur because a man finds 
or is promoted to another job. Whatever the reason, such changes can de- 
grade the performance of the system, and it may be desirable to determine 
the extent to which performance is degraded by such changes in crew com- 
position. 

All we have done thus far is define in very broad terms a system problem. 
Obviously, some of the notions introduced will have to be sharpened a great 
deal before the problem can be subjected to a controlled test. To begin with, 
what precisely do we mean by changes in crew composition? How do we 
want to go about testing the effect of such changes? There are two general 
approaches which might be adopted. First, if the experiment is to take place 
over a reasonable period of time, it would be possible to hold the composi- 
tion of some crews constant and compare their performance with control 
crews—that is, crews which are allowed to vary in an uncontrolled way, 45 
in real life. Second, it would be possible to vary the composition of crews in 
some systematic manner and compare the performance of these crews with 
one another. Crew composition might be systematically varied by position of 
experience level, for example. Varying a crew by position might mean that 
we would limit our evaluation to the effect of changing senior directors OT 
Scope operators or some other class of system members. To vary by experi- 
ence might mean to rotate more experienced men among crews and compare 
their performances with the performances of junior men who are rotated 
among crews. Since it would be impossible to test all thesc permutations in a 
single experiment, let us assume that it has been decided to adopt the lirst 
of the two approaches—that is, to hold some crews constant and compare 
their performances with crews allowed to vary in composition, as in real life 
(uncontrolled), 

. It is unlikely that differences between the two types of crews will be 
significant regardless of inputs. Common sense tells us that the more difficult 
the problem, the greater the likelihood of crew rotatio 
effect on performance. Difficulty presumably is defined i 
to the system. It is therefore necessary for us to specify precisely the input 
conditions under which we Propose to evaluate the crew's performance: 
Assuming that it has been decided to determine difficulty by varying the 
number of penetrating aircraft per unit of time, what does this imply? Are the 
penetrating aircraft to be friendly or hostile or both; and if both, what is the 
ratio of one to the other? How is the phrase “per unit of time” related tO 
difficulty? For example, 100 penetrating aircraft may be distributed within 
any given time period in a variety of Ways, and presumably distribution 
would have a great deal to do with "difficulty." How is input distribution to be 
controlled? One way of dealing with this problem would be to specily the 


n having an adverse 
n terms of the inputs 
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number of tracks appearing during any given unit of problem time, as in 
Table 13.1, but even the "definitions" given in this table are not entirely 
adequate since they do not specify input characteristics which may have a 
significant effect on the performance measures taken unless controlled—for 
example, duration of tracks on scopes, aircraft altitude, or crossing and merg- 
ing of tracks. Furthermore, no rationale has been developed for the values 
used. 

The air-defense problem we are examining has one other key concept— 
namely, degradation. What does it mean to say that the performance of a 
System has been degraded? It means, for one thing, that the measures which 
are used to evaluate system performance have changed significantly. The 
number of cities bombed increases; the number of hostiles detected and de- 
Stroyed decreases; and so on. Some examples of operational definitions for 
these measures appear in Table 13.1. 

There is an important point to be observed in connection with all these 
examples: they differ greatly with respect to adequacy of definition. Both 
the dependent and the independent variables of an experiment should be 
Operationally defined—that is, they should be defined in terms of the opera- 
tions an experimenter must go through in order to demonstrate their meaning. 
The term latency of detection, for example, can operationally be defined as 
the difference in real time between the initial appearance of a target on a 
Tadarscope (t,) and its detection by an operator as indicated by his reporting 
the presence of that target to a plotter (t,); thus, latency = t, — ti. To be 
meaningful, every dependent and every independent variable must be defined 


In this manner. 

, The term "problem difficulty, 
ings, but to be meaningfully manipu 
it must be operationally defined. In th 


" for example, can have a number of mean- 
lated in a system evaluation experiment, 
e example of air defense, difficulty might 
Mean the amount of air traffic to be handled in a given period of time, or the 
mount of noise in the system, or some similar “burden” placed on the sys- 
tem. But even these definitions are far from adequate. Noise varies in type 
and amount, and increasing the number of aircraft alone does not necessarily 
Increase the “difficulty.” To adequately define these—and similar—variables 

© experimenter must be prepared to specify the operations which demon- 


s : 
trate their meaning. 


Once an adequate definition of the independent variables to be manipu- 


lated has been arrived at, a question arises as to the range over which these 
Variables will be presented to the system. Obviously, they fall into two major 
Categories: (1) those which are continuous in nature, such as problem load; 
and (2) those which are essentially discrete, such as crew procedure or 
“duipment configurations. The range of presentation will depend on whether 
Ne variable in question is continuous or discrete. and it will depend upon 
© purpose of the study. There is actually very little by way of guidance 
Uch can be provided here. One fact, however, is clear. The expense of sys- 
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Table 13.1 


Operational Characteristics of the Independent and Dependent Variables 
of the Air Defense Study Discussed in Text 


Experimental Variables Operational Characteristics 
Crew Size = 30 Men i 
z š Crew Composition = No variation in 
EXPE mental number or individuals from exercise to 
Crew exercise. 
composition Crew Size = 30 Men " 
Crew Composition — No variation in 
Control number; individuals selected from those 
available for duty each test day. 
Independent Number of penetrating aircraft appear- 
ing on scopes by type, hostile and 
High friendly, for each 10 minutes of 
problem time. (Total problem 
Problem time — 120 minutes.) 
difficulty I———— 5 
One half the number of penetrating 
Low aircraft appearing on scopes by a 
2 and problem time in high difficulty 
problem. ci 
| Targets bombed/penetrating hostile 
Directly aircraft 
correlated Median latency of initial detection of 
Bertoórmünce with . aircraft : . 
degradation Percentage of friendly aircraft 
Dependent measures of identified as hostile 
degradation z a | 
Indirectly Percentage of bombers detected 
correlated Percentage of bombers destroyed 
with | Percentage of hostiles identified as 
degradation hostiles 


tem experiments precludes the testing of systems in a controlled fashion 
using a large number of treatments or values of a particular independent 
variable. The practice, therefore, is to select independent variables only aftet 
completing limited-control pilot studies at the component and system levels. 

With all these facts in mind, it is now Possible to specify an initial experi- 
mental hypothesis for this problem. If all contingencies are to be met, there 
will still be much planning, definition, and Scheduling to be done; but the 
elements of a testable hypothesis have taken shape: it is hypothesized that 
the system performance of air-defense direction centers is significantly de 
graded by day-to-day changes in crew composition. 'This hypothesis is stated 
positively—that is, it is predicted that degradation will occur. To state the 
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present hypothesis as a null hypothesis, it is only necessary to insert the word 
"not" before the phrase "significantly degraded by day-to-day changes . .. 
composition." 


DESIGN AND CONTROL OF 
SYSTEM EVALUATION EXPERIMENTS 


The Research Program 


It should now be obvious that a large number of hypotheses can be 
generated about a system. Not all these hypotheses will be formulated at the 
Same level of sophistication, and they will differ in their relevance to the 
evaluation mission at hand. The present section describes some of the con- 
ditions which influence the development of an experimental program to test 
Such a set of hypotheses. An experimental program provides direction for 
the experimental effort; it determines the manner in which particular hypoth- 
€ses will be tested; it establishes the order in which hypotheses will be tested; 
ànd, finally, it helps to determine the type of study which will be used to test 
each of the various hypotheses. In many respects the adequacy of the experi- 
mental program will determine the adequacy of the evaluation effort. 

A number of considerations enter into the formulation of such an experi- 
Mental program. While many of these considerations are of a strictly prac- 
tical nature—for example, time, money, and personnel—others concern the 
Content of the hypotheses and the hierarchical relations which exist among 
them. The nature of the studies to be undertaken in order to evaluate the 
System will be determined largely by the content of the hypotheses. Decisions 
about the order in which particular hypotheses will be tested are generally 
determined by the hierarchical relationships among hypotheses. 

There are at least five ways in which system-evaluation questions may 
be studied and answered. These include: (1) using common sense; (2) con- 
Sulting previous research; (3) employing a descriptive analysis; (4) conduct- 
Ing pilot studies; and (5) conducting definitive experiments. 


At the common-sense level of inquiry, the types of 
their scope is relatively limited; and no 
elaborate experimental methodology is needed to arrive at adequate solutions. 
Such problems frequently arise in the design of equipment or the utilization 

System equipment. Let us consider, for example, a device controlled by a 
Movable foot pedal. The foot pedal as it comes from the factory lies with the 
Smooth side down against the floor. When the operator uses the foot pedal 
to control the output of the device, the foot pedal slips. Eventually, the foot 
Pedal slips so far that the operator must stop, feel around for the foot pedal, 
and bring it back to a comfortable working position. The problem is a simple 
One. No elaborate experimentation is required for its solution, and there is 


Common Sense. 
Problems considered are simple; 
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sufficient knowledge already at hand to arrive at a solution once the problem 
is isolated. One could, for example, place an abrasive substance on the back 
of the foot pedal, or one could determine an adequate position for the foot 
pedal and fasten it to the floor. 


Previous Research. Unfortunately, all the problems arising in the cval- 
uation of a system cannot be solved by the application of common sense. 
However, the information necessary for the solution of a problem is fre- 
quently available, though not on a wide enough basis to be considered a 
matter of common sense. Such a state of affairs gives rise to our second level 
or way of inquiry—that is, the utilization of previous research. Problems 
soluble by this approach are usually those which have been encountered in 
the past and, as a consequence, have already been experimentally studied and 
reported in the literature. Generally speaking, the precise problem facing the 
system evaluator has not been encountered before, but problems sufficiently 
similar to it may have been studied, so that extrapolations can be made from 
the results of the previous research. Let us consider the individual who wishes 
to design a console at which a human being will sit to perform a particular 
task. How high should the console be from the floor? What sort of a work 
space should be provided for the individual who will operate the console? 
How much knee room should be provided? Such questions could be asked 
almost indefinitely. Many of these questions have already been encountered 
by human engineers in the past, and many of them have been extensively 


studied. As a consequence of such research, there is a good deal of anthropo- 
metric data available for use in console design. 


Descriptive Analysis. At the two levels of inquiry previously discussed, 
the information necessary for the solution of the problem was available to the 
system psychologist. In this and in the next two cases, the information the 
system evaluator needs is not available and becomes so only through experi- 
mentation. When the methodology of descriptive analysis is employed. 4 
minimum of experimental controls are required; the fundamental objective is 
an empirical description of the system, its data flow, and its response times: 

Three types of information are commonly collected for a descriptive 
analysis: (1) through-put times for man-machine operations, including those 
applicable to (a) individual components, and (b) sequences of components 
where the output of one component serves as an input to the next; (2) failure 
rates in machine operations; and (3) frequency and content analyses of SYS" 
tem communication links. Since the collection and utilization in system evalu- 
ation of the first two of these types of information is relatively obvious: 
further discussion of them is not required. This is not true, however, of the 
third type of information collected for descriptive analysis, frequency and 
content analyses of communica'ion links. Such analyses of communication 
links are commonly conducted on the communication channels linking Me” 
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with one another. or linking men with machines. The technique most com- 
monly employed for this purpose consists of making an exhaustive listing of 
all possible man-to-man communications and all possible man-to-machine 
communications and then observing the system in operation, tabulating the 
frequency with which the different types of communications occur. An illus- 
tration of such a tabulation is shown in Table 13.2. Once such a tabulation 


Table 13.2 


An Illustration of a Link Analysis? 


Men 
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Officer | | 
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“Fighter Director 
(Fight. Dir.) 
| | 


VF Radar 
Operator 
(VF Oper.) 


VC Radar a — 


———| 3 9 


Air Plot” 


“Radio Desk. 
E i | | | 


Machines 


VF Radar 


“VG Radar No.2 | | 
| 
| 


Director Repeaters 


en and machines in the CIC of the 
n and machines are genuine. 
of the author and the publisher, 


and between m 
ttacks. Data is fictitious. Me 
959), by permission 


Linkages between men : 
E Louisville during air attic 
able and note from Chapanis (1 


506 - Evaluating Performance in Simulated Environments 


has been accomplished, it can provide considerable insight into the scope of 
the communication of one man or one machine with other men and komen 
in the system; it also provides clues regarding the communication aimar = 
being placed upon particular links. Such results can be used to pisn a 
arrangement of men and machines, determine the number of communicatio 

channels required to handle various communication loads, and help to pies 
ways and means of accomplishing load-sharing between individual system 
members. 


Pilot Studies. The pilot study is an almost indispensable tool for any 
developmental activity; it is generally used as a device to help collect informa- 
tion in those situations where very little or no information about a system IS 
available. Typically, pilot studies are used to obtain some initial indications 
of the kinds of measures which should be taken, to discover where specific 
types of experimental controls should be applied, and to help establish the 
important independent variables for further study. . 

One way to characterize the pilot study is to assert that it is an experi- 
ment in which the experimenter is more concerned about making a so-called 
Type II error than making a Type I error. Technically speaking, a Type H 
error is made when the null hypothesis is erroneously accepted. The experi- 
menter, in other words, accepts the hypothesis that a variable is not significant 
when it actually is. In the pilot study, one is interested in accepting as many 
potential variables as possible in order to be sure that all pertinent variables 
are, in fact, considered; consequently, the aim is to minimize the likelihood 
of committing a Type II error. At this stage, the experimenters are concerned 


with excluding those variables Which obviously have no eflect on the phe- 
nomena being studied. 


esigns employing smaller num- 
bers of subjects and an acceptance, at the outside, of a 5 percent level of 
confidence, and frequently, results which are significant at between the 5 per- 
cent and the 10 percent level of confidence. If the requirement for the accept- 
ance of the null hypothesis is less stringent than 10 percent, however, there is 
considerable danger that the experimenter will pursue false leads in subse- 
quent controlled experimentation and will consequently waste valuable Te 
search time. In general, it should be remembered that a pilot study is 
exploratory in nature; it is an instrument used to search for valid measures, 


to explore potential independent variables, and to establish the experimental 
controls to be exerted, 
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Definitive Experiments. The nature of the problem in this case is some- 
what different from that which we have just considered. When definitive 
experiments are used, it is assumed that the important variables in a particu- 
lar situation are known and that something is known about the controls 
Which must be exerted and about what measures must be employed in the 
Situation. The purpose of definitive experimentation in system evaluation is, 
first, to check the results of pilot studies in order to be sure that the kinds 
of variables which have been selected as a consequence of conducting these 
Studies are, in fact, those which do operate in the particular context. A second 
Purpose of the definitive experiment is to determine the nature of the rela- 
tionship which holds between important independent variables and measures 
Of system performance. A third purpose of the definitive experiment is to 
determine the limits of a relationship which may hold between the chosen 
Measures and given independent variables. It is seldom the case that a rela- 
tionship between a given independent variable and a given measure holds 
Over all ranges of the values of these measures and variables. It is more often 
the case that a given relationship will hold only within certain limits. Finally, 
the definitive experiment is conducted to determine the nature of the inter- 
actions which occur between the various independent variables operating in 
the system. 

In the definitive experimental situation, the experimenter is much more 
Concerned with the commission of a Type 1 error than a Type H error. A 
Type I error involves the rejection of the null hypothesis when it is true— 
that is, the experimenter asserts that a difference probably exists, when this 
iS not the case. As a consequence of this fact, the levels of confidence a 
are accepted for significance are considerably more stringent. In addition, t : 
Usual context of a definitive experiment is one in which there is more an 
More concern with the reliability of the results. More time, more effort, more 
Subjects, and more money are invested in a definitive vi ancien v : 
Pilot study. The definitive experiment is a reasonable extension : the ps 
Study, The relative concern with the Type II error in the pilot study must be 


Compensated for by a concern for a Type I error in the sca e 
€ acceptance of a lower level of significance under the relatively limite 
T H H 


Conditions of a pilot study must be checked in a more carefully controlled 
eXperimental situation. For a more thorough treatment of cs gend T I 
and Type II errors and their relationships to levels of significance, the reader 


'S referred to Lindquist (1953). 


Experimentation with Systems 


The experimental methodology used in the conduct : drei a 
“valuation experiments reflects the kinds of constraints which almost inevita- 
ly insu this type of experimentation In the present section, three of 
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the most important of these constraints will be discussed. The first of these 
has to do with the kinds of experimental control exercised in systems research; 
the second concerns the collection of experimental data; and the third con- 
cerns problems in the area of experimental design. 


Experimental Control. Two characteristics of a conventional experi- 
ment are worth noting, both having to do with the reliability of experimental 
results. One concern for reliability arises as a consequence of random varia- 
tions among subjects when a large number of subjects is being utilized. The 
university psychologist studying a problem in learning in human beings, for 
example, generally has at his disposal a large number of potential subjects 
from introductory classes in psychology. When large numbers of subjects arc 
used, it is unlikely that the effects of individual differences will materially 
influence the results of an experiment. There is, however, a difference between 
this type of situation and the one in which system evaluation experiments are 
typically conducted. Subjects in system experiments are the groups of people 
operating the system, and it is usually very difficult to get a large number of 
such groups or crews. The time-honored technique of randomizing individual 
differences, therefore, does not generally apply in the evaluation of large 
systems. 

A second characteristic of the conventional experimental problems con- 
cerns the range of external variables which must be controlled and the tech- 
niques which are used to control them. Let us consider the problem of human 
interactions. If the experimenter in a university setting is concerned with the 
interactions of human beings during an experiment, the experiment can be 
conducted so that only one subject is tested at a time. The very nature of the 
phenomena studied in the typical system experiment requires that human 
beings interact with each other. As a consequence, human interactions will 
always occur. The control of the environment in which the human being 
performs in a system experiment is complicated by a second problem. Indi- 
vidual system members are almost always involved with a machine; the 
machine contributes to what the man does, and the man contributes to what 
the machine does. Thus, in any man-machine system, there is a large 
number of interactions between men and machines, These two sources of 
variations in performance are not necessarily objectionable in and of them- 
selves. They are characteristic of all systems and should be included in syste™ 
studies. Problems arise, however, when human beings become involved in 
tasks not directly related to their system performance hi when machines mal- 
function. Such difficulties translate themselves into problems of attention spar 
motivation, and morale in the case of a human being, and into problems of 
reliability and failure in the case of the machine. The fundamental problem 
is to keep the human being (or the machine) task-oriented 

The environment of the system ex 


à à ` periment is complicated in another 
sensc—that is, there is considerable inte 


raction between related information" 
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processing tasks. This gives rise to the problem of description of the stimulus. 
The input to a system can be described in considerable detail and with con- 
siderable accuracy. As soon as this input information has gone through the 
first of a series of processes, however, the experimenter loses the opportunity 
to describe and specify precisely what the input to subsequent processing 
tasks actually is. 

A third characteristic of the system context which creates control diffi- 
Culties is the existence of a number of alternative responses in the system 
context. In the typical psychological experiment, the experimenter is con- 
cerned with one. or at most a few, of the responses of his subjects. Further- 
more, the experimenter often constrains the experimental situation in such a 
way that the responses of the subject are directed toward the particular aspects 
of behavior of interest. In reaction-time experiments, for example, the task 
of the subject is specifically oriented toward a particular form of behavior. 
Other examples of the same sort of attitude are to be found in nonsense- 
syllable learning, in experiments involving anagrams, and in maze learning 
in human beings. It is characteristic, then, of the usual experimental situation 
employed in psychology for the experimenter to be concerned with a partic- 
ular aspect of a subject's behavior. This is not the case in a system context. 
There is, of course, a preferred mode of behavior, and appropriate modes 
for interaction are provided for the members of a crew. But there is no 
guarantee that crew members will utilize these preferred modes of interaction. 


In most systems situations, modes of communication are provided for those 


Channels the system designer wishes to have employed. It is not always pos- 
f crew members to such modes of 


Sible, however, to restrict the behavior o 
communication or to restrict communication to specific individuals. Typically, 
When an individual wishes to talk to someone with whom he has no official 
Connection, he simply raises his voice and completes the communication. 
These “extracurricular” modes of communication serve two purposes. On the 
One hand, they contribute to the lack of precise control in the experimental 
Situation, and in this respect they are an unfortunate occurrence. On the 
Other hand, in the development of a system, it is just exactly this sort of 
Occurrence that the experimenter often wishes to observe. Frequently, such 
“extracurricular” communications ( particularly if they occur with any great 
frequency) are a source of information about inadequate communication 
facilities within the systems context. If one is in the developmental phase and 
One is concerned with the development of measures or techniques for experi- 
Mental control, or with the isolation of important independent variables, he 
may welcome such occurrences. On the other hand, if the experiment is rela- 
tively precisely controlled and such acts occur, they will be anything but 
Welcome. 

A final consideration with respect to control concerns the problem of 
Variability of performance among human beings. Given a crew of individuals 
to man a particular system, individual abilities differ over a relatively wide 


510 - Evaluating Performance in Simulated Environments 


range. In itself, this is not a particularly surprising observation. When con- 
sidered in conjunction with the fact that information-processing systems fre- 
quently involve a sequence of processing events, however, it can be scen to 
lead to considerable variation in system performance, as well as lack of con- 
trol in the experimental situation. One person may be assigned to do a par- 
ticular job and he may do his job very well; another individual —presumably 
possessing similar skills—may perform the same task very poorly. Thus, the 
output of the component in the two different situations is quite different, and 
the input to a subsequent component is not the same. As a consequence, there 
is a rcal problem in the selection of individuals to fill particular positions on an 
experimental crew. One method of dealing with this problem is to attempt to 
match individuals from one crew to individuals from the next on the basis 
of skill. However, this is only practical in those instances in which there is à 
large number of individuals to choose from and the crews are relatively small. 
Another solution to the problem is training; and indeed, it frequently happens 
in the system context that one spends as much time preparing individuals to 
serve as subjects as one spends utilizing them in evaluation experiments. 

These considerations imply that the problem of control in a systems 
experiment is considerably greater than the problem of control in the more 
conventional psychological experimental situation. This difficulty has led to 
two attempts to solve the problem of control. One has been the search for 
"system measures" which would transcend the difficulties mentioned, and the 
second has been the attempt to break systems up into subsystems, to study 
the subsystem independently and then to put the subsytems together in a 
selective fashion. Both these techniques are useful and both are currently 
used in present-day systems evaluation. 


Data-Collection Problems. As we have pointed out, the system eval- 
uation experiment may involve as many as 15 or 20 different types of response 
measures. There are three consequences of this state of affairs worth discuss- 
ing. The first and most obvious of these is that a relatively large number of 
monitors of system performance are generally required to collect the neces- 
iderable amount of staff training is required. 
ection and reduction task creates a second- 


higher-level system. A set of individuals who are concerned with the collec- 


collection, constitute another 


l ! kind of system—in effect, a metasystem. The 
same variables which influenc 


e the behavior of the data-processing system of 
collection system which moni- 
most pervasive variables affect- 


trained to recognize and collect 
collecting this data they do not 


—Robert H. Davis and Richard A. Behan - 51 


become overloaded. When data collectors become overloaded, there is an 
interaction between the system that is being observed and the system doing 
the monitoring. In such a situation, the response measures being collected are 
no longer related solely to the system under observation, but reflect instead 
the behavior patterns of both systems. 


Experimental Design Problems. There are two considerations in the 
Selection of an experimental design for system experimentation. The first of 
these has to do with the problem of replication, and the second has to do 
with the independence of measures. These two problems are, of course, inter- 
related; but, for the purpose of the present discussion, they may be considered 
Separately. 

The conventional experimental design generally utilizes different sub- 
jects to counter the problem of variability. As indicated previously, in a 
System context individual human beings are rarely if ever subjects; crews are 
the subjects for experimentation. Since crews consist of relatively large num- 
bers of human beings, there are problems in obtaining a satisfactory number 
of crews, A second consideration which enters here, and one which frequently 
Serves as a basis for the relationship between our two points of discussion, is 
that the difficulty of obtaining crews makes it necessary to use a particular 
Crew over a series of experimental trials. In consequence, it is seldom possible 
to deal with independent measures in a typical system experiment. The prob- 
lem of replications is handled by utilizing a crew on more than one occasion, 
and assumptions about independence of measures cannot be met. " 

A cursory examination of the considerations desenteng in the a E 
Paragraphs will convince the reader that the system caer sns t i 
More sophisticated types of experimental design. Only a few su jects pem 
are available, and hence very little use of between-subject variance can be 
made as a source of the effect of independent variables. Since vE m with 
small numbers of subjects and are very seldom able dine the | rego 
Subject variance as the source of the effects of an experimenta nor e, the 
Conventional experimental designs as they appear 1n the tetra aeri 
are not particularly useful. The most useful — pes AET 
factorial design in which the effects of the experimenta Pan: dos : 
Out of within-subject variance, and in which the eere ecc eii um 
quently split between the between-subject variance anc the j 


Variance (Lindquist, 1953). 


Crew Preparation and Motivation 

n the area of psychology assumes some subject 
nclude extensive preliminary training or 
ubjects a set of prepared instructions. 


Most experimentation i r 
Preparation, Such preparation may ! 
it may only involve presenting to the s 
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The degree of preparation will depend on the design of the experiment. Such 
preparation is aimed at establishing appropriate sets and attitudes and insur- 
ing that subjects have achieved a level of proficiency appropriate to the 
experiment. In the case of system research, this problem is compounded many 
times. Each individual in the system must either be trained to the level of 
skill required after arrival at the experimental site, or he must bring such 
proficiency to the experimental situation. In either case, the experimenter is 
faced with a problem. If it can be assumed that the subject can bring the 
requisite skills to the experimental situation, then the problem is one of 
selection. If the subjects cannot acquire the skills prior to experimentation, 
the problem involves setting up appropriate training procedures after arrival. 

As we have emphasized, the term "subject" in system research implies 
à crew, and crews consist of many individuals. When such is the case, the 
treatment of these individuals becomes an extremely complex problem. Since 
crew members will live in the experimental area for many days, there are à 
number of important questions which have to be dealt with. What kind of 
contact should crew members have with the experimenters? What should 
they be told about the objectives of the experiment? How much information 
should they be given about the techniques for generating controlled inputs? 
What provisions must be made for sickness, accidents, and unexpected shifts 
in crew composition? What level of motivation should be maintained, and 
how? 

The results of an experiment can be seriously affected by the motivation 
and morale of the crews manning the system. Despite the effort which psy- 
chologists have expended on the study of motivation, there is little in the 
literature to aid the system psychologist. 

Some very tenuous generalizations can be drawn from the experience of 
the authors in the area of system research (Behan, Davis, and Pelta, 1959)- 
In general, the longer a given crew must Participate in an experiment, the 
greater the drop in motivation, Similarly, if some unusual circumstances sur- 
round a crew's participation in an experiment, there is a greater likelihood 
that motivation can be maintained. Bringing crews to a special laboratory for 
System studies essential to the national welfare, and introducing special incen- 
tives such as competition between crews, have been used to maintain motiva- 
tion. Some experimenters have, in fact, systematically cultivated an “aura of 
mystery" in order to stimulate crew motivation. 

Many such techniques used to enh 
so-called Hawthorne effect. For discussi 
to Brown (1958) and Chapanis (195 
Hawthorne effect depends on the generalizations one wishes to draw from 
the experiment. Where motivation has been artificially increased by manipu- 
lating the social (laboratory) climate, it becomes extremely difficult to gen- 
eralize experimental results to everyday field Operations. On the other hand, 


ance performance are variants of the 
on of this effect, the reader is referred 
9). The advisability of exploiting the 
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if one wishes to know what kind of performance can be expected with opti- 
mum motivation, such procedures may be justified. 


Changes in System Behavior as a Function of Practice 


Change of behavior as a function of practice is a characteristic of systems 
as well as of individuals. The conventional response of the psychologist to 
evidence of improvement in performance with repeated trials is to assert that 
learning has occurred, and many psychologists have accepted learning as a 
valid explanation of this finding. However, it is unlikely that all instances of 
à change in system performance as a function of practice can be explained 
by learning in the conventional sense of the word. While it is probable that a 
Certain amount of learning, and consequent improvement in system perform- 
ance, does occur among the men who work in information-processing systems, 
it is also quite probable that a large share of the change in system performance 
is due to the adoption of more fortunate operational procedures by men 
Working in the system (see Chapter 11). Evidence for this latter assumption 
Comes from the fact that: (1) such changes in performance are observable 
in systems which have been operational for long periods of time, in which, 
Presumably, the task proficiency of the men working in these systems should 
have reached a maximum; and (2) one can often find decreases in activities 
Which correlate with observed increases in other measures of system perform- 


in one set of studies (Chapman et al., 1959) it was ob- 


Served that the experimental crews displayed a consistent increase in the 
and intercepted. This improved 


Number of hostile aircraft which were detected 

Performance was, however, accompanied by a decrease in the number of 
friendly aircraft which were processed by the crews. Crews were not proc- 
essing more aircraft; they were merely being more selective in aircraft 
Processing. , 

The use of certain procedures may have a detrimental rather than a 
beneficial effect on system performance. System input and information- 
Processing procedures are almost invariably designed to deal with d clean 
"put, but a not infrequent occurrence is for an information-processing Low 
to experience noise in the input to the system. When a system experiences 


noise į i i onnel frequently want rather extensive— 
n upervisory persi y w 
ie , bout the noise itself. The use of such 


sonnel increases the information- 
nd thus results in a decrease in system per- 


ance. For example, 


Processing load on the system a 
o 


514 - Evaluating Performance in Simulated Environments 


obviously refers to the fact that some system-performance measures show 
improvement as a result of repeated trials (experience). The source of this 
improvement, however, is not always obvious. It may result from the com- 
ponent-skills training which accompanies practice in a system context; it may 
also result from the adoption of special procedures; or it may reflect the 
establishment of priorities by the system. 

In the final analysis, all so-called system learning must reflect changes 
in the behavior of individual system members. On the other hand, it may be 
valid to continue to draw the distinction between system learning and individ- 
ual learning, because the former can only be learned in the context of the 
system. One may, for example, train a man outside the systems context to 
read a radarscope and report his observations to another individual who plots 
them on a vertical board. But there is a great deal of procedure learning 
which takes place between these two individuals which cannot be taught in 
isolation from the system. The radarscope operator will learn, as a result of 
his system experience with the plotter, something about the speed with which 
the plotter can handle the traffic which he calls to him. He will learn some- 
thing about his idiosyncrasies and weaknesses, and as a result of knowing 
these, he may be able to adjust his reporting speed and format to insure the 
accurate and timely transmission of the information. Thus, the distinction 
between systems and individual learning can be a useful one, even though its 
bases have not yet been fully explored. 
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DURING THE COURSE OF HIS WORK IN SYS- 
tem development, the psychologist may be called upon 
to participate in activities extending from the original 
conception of a system to its testing and use, as pre- 
vious chapters have shown. As a scientist, he is called 
upon to make many decisions throughout this Process, 
some of the most important of which are decisions 
i nethods to be employed in answering system questions. 
ja general, because of his scientific training, the psychologist yc ine 
well acquainted with the methodology of obtaining behavioral evi we 
which support hypotheses derived from theory. He is likely to be less in 
acquainted with the requirements of "psychological engineering ww 
frequently generated by systems problems. For this reason he may find ei 
self in a situation of conflict when it comes to a choice of methodology. m 
may even be unaware that he faces an option. Such conflict need not inr 
when there is adequate understanding of the methods appropriate. Siep 
neering, and their specific relationship to those used by the scientist ! 
answering questions of a theoretical nature. T" 
The discussion of this chapter is one which compares and contrasts t 
methodologies appropriate to two kinds of "system"—the conceptual system 
of the scientist, and the empirical System of the engineer. The primary aa 
pose of the former is seen to be understanding, and of the latter, con 
When the implications of this distinction are traced, it becomes evident wr 
the system psychologist needs to recognize a number of methodologica 
options in the planning and conduct of his work. These range all the way 
from the initial formulation of the problem through the stage of analysis tO 
the drawing of conclusions and the utilization of research findings. The 
varieties of these options, and how they influence the research activities of the 
investigator, are explored and illustrated by means of examples. " 
There is provided here a most important philosophy and point of view 
toward the entire process of System development, and toward the role of me 
psychologist in the enterprises described in previous chapters of this book. 
In particular, the present chapter may be conceived as laying the foundation 


for some truly systematic methodologies for Psychotechnology, which will 
hopefully be developed extensively in years to come. 
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THE SYSTEM CONCEPT 
AS A PRINCIPLE OF 
METHODOLOGICAL DECISION 


John L. Finan 


Most of the systems with which the preceding chapters of this book have 
been concerned are examples of directly observable, "natural" groupings. It 
Will be useful here to consider such systems as special cases that fall under a 
Concept of broader scope. A system is essentially a way of conceptualizing 
experience, according to which the components of an organized grouping 
interact to achieve a designated purpose. The objective of the present chapter 
iS to examine some of the ways in which the system orientation, particularly 
as it reflects the pragmatic principle, provides explicit criteria to which deci- 
Slons of rescarch design may be referred. = 
A fundamental distinction first needs to be made between empirical and 
Conceptual systems. The theoretical scientist who sets. understanding of phe- 
Nomena as his immediate goal is mainly concerned with conceptual systems, 
Y contrast, the engineering scientist whose immediate interest is in controlling 
"Dature" deals with systems of an empirical type. Whether research is thus 
Problem-oriented or product-oriented will govern the decisions made at vari- 
US critical stages of inquiry. The discussion which follows is organized 
around eight design options that are considered applicable to these two kinds 
research, 

T An initial option is taken at the stage 
Search problem. In theoretical research, 
Ore controllable context, and the variables 


of selecting and formulating the 
the problem is transposed into a 
jnvolved are translated into con- 


317 
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ceptualized dimensions. In contrast, the prime requirement for the results of 
engineering research to be relevant to practical goals restricts this latter oP 
of study to situations and variables that closely simulate the complex O 
operational conditions. ' 
A. second option concerns the use of analogies in research. The age 
tory model of theoretical inquiry is essentially a symbolic Se 
observations, while the forecast formula of engineering psychology can e 
considered an empirical summary of results. A related distinction is made 
between validity of theoretical research—the correspondence between a con- 
cept and germane phenomena—and fidelity of engineering research—the de- 
gree of relationship between forecaster and criterial terms required for a 
specified practical purpose. f 
A third option is taken with respect to the differential role of hypothesis. 
In theoretical research, hypotheses are explicitly linked to a model, and if 
experimentally corroborated, make possible attribution of effect to cause- 
When employed in engineering psychology, hypotheses may serve to suggest 
the content of forecaster and criterial terms, and may, post hoc, be used tO 
interpret the observed relationship between them. The hypothesis-testing and 
demonstration types of experiment exemplify this contrast. T 
A. fourth option is taken in dealing with the problem of the variability 
of observations. Causal attribution depends, in the ideal, on the possibility of 
rigorously controlling all relevant experimental conditions other than the one 
manipulated. Forecasting, however, depends on representing with maximum 
fidelity whatever sources of variability may operate within the criterial situa- 


tion. This is the important difference between systematic and representative 
design. 


A fifth option is taken in order to defi : 
research, units are selected for the purpose of demonstrating behavioral d 
formities; in engineering research, the requirement is to define a unit tha 


s à ; ional 
proves manageable for producing or forecasting a particular operationa 
system. 


; . soretical 
ne units of analysis. In theoretic’ 


A sixth option is taken with reference to the criteria of acceptable 
ence. Statistical hypothesis-testing is considered more appropriate to the 
onstration of model relationships, while statistical estimation techniques 
are deemed more suitable for forecasting to a criterion. The appropriateness 
for engineering studies of the conventional .05 probability level for accept 
ance of findings will subsequently be questioned. 

A seventh option is taken when conclusions of research study are 
extended to new situations. In theoretical research, generalization proceeds 
by demonstrating the extensibility of the model dimensions to the new COM 
ditions. Limited engineering generalizations can be b. 
populations and guessed interclass relationships. 

A final option is taken in“connection with utilization of research out 
comes in practical situations. Results obtained under the pure conditions 9 


inferi 
dem 


pde m 
ased on inference fro 
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the laboratory yield abstract predictions which may have implications that are 
adaptable to operational systems. Engineering studies yield forecasts, which 
constitute a direct and immediate basis for action. 


SOME IMPLICATIONS OF THE SYSTEM CONCEPT 


[1] 


The term system refers to a set of components organized in a way that 
tends to constrain action toward a specified end. The particular value of an 
input-output relationship which the system is designed to produce defines 
the purpose of the system. The purpose is what gives integrity to the system. 
A purely conceptual system—for example, a non-Euclidean geometry—is 
Organized under the theoretical goal of explicating the logical consistencies 
Obtaining within the postulate structure. Or, to consider another kind of 
example, a particular man-machine system is designed and developed in 
Telation to the goal of defeating an enemy under a specified set of conditions. 
It is important to note that the concept of system poses explicitly, and as its 
central problem, the role of purpose or value in relation to the design and 
methods of inquiry. The pragmatic principle, according to which purpose or 
Value is regarded as a criterion of valid knowledge, lies at the root of the 
System concept. 

. Organizations may consist of concepts or of observed events. A distinc- 
tion has already been made between conceptual systems, to which the ques- 
tion of homology with observed nature is irrelevant, and veridical systems, 
Which are accorded the status of observed events. Characteristically, in 
theoretical science, a set of concepts is employed in an analogical relationship 
With a set of observed events, and the degree of correspondence is noted. 

bstractions of this theoretical sort are, within the logic of systems, explicitly 
"ecognized as tools invented to further the purpose of understanding, rather 
than as “ultimate principles” which are “discovered” to exist in nature, 

Integrating what has been said up to now about purpose with the 
©Onceptual-empirical dichotomy puts us in a position to grasp the main point 
Of the present discussion. Research decisions will differ in fundamental ways, 
depending on whether the primary purpose of a given research inquiry is to 

vild a conceptual system that represents the logical relationships obtaining 
Within a given range of phenomena, Or whether it is to produce or forecast 


an emm. 
^ efficient natural system. ditionis: 

th Within science broadly understood, two purposes are istinguishable: 
et 


heoretical, which seeks to understand, and the engineering, which seeks 

Control, *On what principle does the system work?” is the question to 
Which the scientific theorist seeks an answer; “How can a system which is 
wtended to meet a specified objective be produce 


d or improved?” is the 
Sha dice a 
ngincer’s question. To formulate laws of a highly a 


bstract type which sub- 
Sume broad areas of phenomena beyond those brought under immediate 
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observation is the goal of theoretical science. To develop or to improve a 
machine, or other natural function, in accordance with some practical end is 
the goal of engineering science. Stated in another way, theoretical research is 
concerned with the entire range of output variation in relation to changes in 
input; engineering research restricts interest to those input factors that yicld 
a particular output judged to be of practical value. Within the area of human 
behavior, theoretical psychology is concerned, for example, with laws of 
learning, motivation, and attitude change, while psychological engineering 
deals with such matters as man-machine interactions, intelligence and aptitude 
testing, clinical diagnosis and therapy, control of attitudes and beliefs, and 
the improvement of educational programs. 

In order to make the distinction between these two approaches to 
research more concrete, two experimental studies— Miller's investigations of 
conflict behavior (1959) and MacCaslin, Woodruff, and Baker's work 
(1959) in developing a course of training for Armor trainees (SHOCK- 
ACTION VI)—will be followed as paradigms throughout the discussion. The 
summary accounts of these studies given below are intended to emphasize 


those features that will become relevant to points discussed subsequently 17 
the chapter. 


A Paradigm of Theoretical Research 


A closely integrated program of investigation of approach-avoidance 
behavior conducted by N. E. Miller serves as the reference study for the 
theoretical approach to psychology (Miller, 1959). The stated goal of this 
research is to formulate the laws of conflict-behavior. 5 

As a first step, a limited postulate system, based in large part on prin- 
ciples borrowed from Hull's learning theory, is set forth. Hull's principle 9 
the goal gradient is, for example, the source of the assumption that the eerie 
the subject is to a goal, the stronger is the tendency to approach it. A paralle 
postulate assumes that the closer the subject approaches a frightening stimu- 
lus, the stronger is the tendency to avoid it. Next, deductions rigorously 
derived from the set of postulates are explicitly stated. Miller hypothesizes 
that under certain conditions the subject should approach the goal and stoP 
at the point where the strength of the approach and avoidance gradients are 
equal. Or, to cite a second example, it follows deductively from the postulates 


that when the strength of hunger is increased, the subjects will approac 
more closely to the goal. 


As Miller points out, in many studies of the present type it is strategic 


to employ lower species in simple situations, which can then be rigorously 
controlled, for first roughing out the approximate lawful relationships. ^C 
cordingly, he employs hungry albino rats trained to run down a short straight 
alley to obtain food. In a later phase of the study the animals are giver 
electric shock at the goal position. The independent variables manipulate 
are selected on the basis of their unitary and readily specifiable character 
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as, for example, hours of food privation and distance from the point of rein- 
forcement. Dependent variables, such as an animal's strength of pull on a 
string harness, and its speed of running, are also chosen to meet requirements 
of objectivity and unidimensionality. Between these two terms, and explicitly 
related to each one, stand the intervening variables. In order to gain generality, 
the empirical variables are translated by means of operational definitions into 
assumed dimensions of the intervening variables. Although admittedly limited, 
Miller's operational definitions attempt to perform the crucial job of relating 
the systematic variables (parameters) posited in the conflict model to specific 
experimental conditions. Examples are the assumed positive relationship be- 
tween the strength of the animal's response tendency and the strength of pull 
When temporarily restrained, or that between greater strength of electric shock 
and greater strength of the fear drive. Sources of variation presumed to be of 
minor importance—such as duration of shock, or amount of food—are 


either held constant or randomized. 
The results of Millers preliminary experimentation (see Figure 14.1) 
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The diagram presents an abbreviated formulation to 
d by Miller and his associates working with white 
rats in the simple conflict alley. Listed on the left side are the independent 
(1-7) and on the right, the dependent variables (8-10). Although these 
variables are closely anchored to specific experimental operations, it will 
be noted that they are stated in general (conceptualized) terms. Variables 
LIV shown in the center of the diagram are the intervening variables, or 
model parameters. The arrows representing functional relationships between 
the terms are labeled with letters which indicate the definitions (small let- 
ters) or postulates (capital letters) to which they refer in Miller’s system. These 
relationships illustrate the operationally defined nature of the concepts 
included in the model. Variables represented by dotted lines, reduced body 
weight and habituation to the feeding cycle, and others omitted from the 
diagram—for example, amount of food and'size of bars on grid—were con 
trolled by holding them at a constant value, or by randomization, in the 
series of experiments. (Adapted from Miller, 1959.) 
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tend to corroborate his model of conflict behavior. To consider one of the 
simpler instances, the data confirm the hypothesis that animals placed nearer 
the end of the alley where they had received shocks would start to run faster 
than those placed farther from the shock position. A more complex deduction, 
also confirmed, required that the animals move closer to or farther from the 
goal, depending jointly on strength of shock and degree of hunger. In a pre- 
liminary series of experiments, 10 out of 11 deductions were confirmed ier 
small groups of animals at the conventional level of significance. The fact t : 
these experiments were specifically designed to validate deductions rigorously 
drawn from the system of postulates is important. Further, the fact that the 
outcomes were predicted from six postulates (in addition to coordinating 
definitions) is impressive evidence in favor of the functional unity of the 
hypothetical constructs posited by the model. i 

The generality of Miller’s theory of conflict may be seen in his successfu 
extension of the simple model to a wide variety of conflict phenomena beyond 
those initially investigated. The original assumptions, for example, have appli- 
cation to avoidance-avoidance and double approach-avoidance conflict. Fur- 
ther extension of the scope of the postulate System to cover aspects of the 
phenomenon of displacement, projective tests, psychotherapy, combat, eco- 


nomic competition, and even international relations, remains to be demon- 
strated. 


A Paradigm of Engineering Research 


An experimental study in curriculum development entitled SHOCK- 
ACTION VI offers a representative example of the engineering approach te 
psychological problems (MacCaslin, Woodruff, and Baker, 1959). This 
research has been previously described (Chapter 9), but will be reviewed 
briefly here to emphasize its methodology. 

The problem situation for which a solution was sought arose out of pet 
formance deficiencies of armor crewmen. The objective of the task was t? 
develop by experimental means an improved program of training for tank 
gunners, drivers, and loaders, as judged in terms of the proficiency level of 
their skills and knowledges. . 

The ultimate criterion for this task, as with most military research, 1$ 
successful performance in combat, which constitutes the set of conditions tO 
which a forecast based on the outcome of the study must apply. Presumably; 
training objectives established by armor experts and detailed task descriptions 
drawn up by researchers reflect the requirements of combat closely enough tO 


provide a basis for developing the curriculum, as well as for constructing 
proficiency tests to determine its effectiveness, 


The experimental treatment carefully limited the content of the course tO 
materials judged directly relevant to the essential functions of the crewmen, 
emphasizing training objectives and combat realism. To maximize the effec 
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tiveness of training, several empirically founded rules of learning were incor- 
porated into the developed program: (1) making explicit the relationship of 
tasks to one another and to the program as a whole; (2) insuring under- 
standing by simplification of materials; (3) emphasizing learning by doing; 
(4) giving knowledge of progress to the student; (5) increasing motivation 
by means of demonstrations, orientation sessions, and compulsory make-up of 
deficiencies. 

In addition to differences in content and training procedures already 
mentioned, the experimental program included several departures from the 
standard administrative procedures employed by the Armor Training Center: 
(1) schedule of training by platoons rather than by companies; (2) formali- 
zation of make-up and remedial training; (3) changes in the manner of 
utilizing instructors; and (4) provision of additional equipment and materiel, 
including illustrated manuals, tank instruments used as training devices, and 
modified firing and driving ranges. The altered administrative procedures were 
deemed to fall within the limits of feasibility and efficient utilization of man- 
power and materiel resources available under actual conditions of training. 

The experimental training program was administered over a period of 
Six weeks to an available company (N — 160) that had been constituted by 
routine administrative procedures. At the end of this period, performance 
tests were administered to the experimental company, as well as to an equiva- 
lent company that had simultaneously completed the conventional eight- 
Week training course. Variables other than those comprising the treatment 
Were equated or otherwise controlled. Performance of the two companies was 
Compared by means of a specially constructed test battery consisting of some 
400 items, grouped into 21 categories of subject matter. Each subject-matter 
test was judged to sample an aspect of performance considered essential to 
efficient performance of a tank crewman because of its frequency or impor- 


tance, 


The results demonstrated that the over-all proficiency of armor grad- 


uates trained in the six-week program was slightly higher than that of grad- 
Uates of the eight-week program in the essential skills of tank gunner, driver, 
and loader (see Figure 14.2). The principal gains of the experimental pro- 
Bram were, first, a somewhat higher level of trainee proficiency; and second, 
a saving of 25 percent of the eight-week training time consumed by the 
Standard program. An appreciable reduction In gasoline and ammunition 
requirements also resulted. The principal costs were (1) an increased require- 
ment for training aids and literature, as well as for tank instruments; and (2) 
an increased requirement for holdover students to serve as assistant instruc- 
lors, A comparison of the two programs in terms of gains and costs was 
Considered to justify the recommendations that the experimental program be 
adopted for operational use in the Armor Training Center where the study 
Was conducted. The developed program wes also recommended for use in 
Other, similar Army programs. As à final point in connection with this study, 
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it is noteworthy that a precautionary statement is included warning the user of 
the program against assuming that departures from the recommended pro- 
cedures would be effective if isolated from the context of the total program. 
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E COMPLEMENTARY NATURE OF THEORETICAL 
AND ENGINEERING SCIENCE 


The two experimental examples have been presented in detail mainly 1” 
order to emphasize differences between the theoretical and engineering ap^ 
proaches. There is a risk, however, in regarding the two positions as mats 
than conceptual extremes, for although in some ways they reflect marke 
differences in purpose and procedures, in others they are complementary; the 


"m s l 
activity contribute to the ultimate £02 


i s h 
» the more immediate goal of prid 
approach becomes the longer-term goal of the other; the abstract laws of pur 


science find eventual application to particular situations of practical conse 
quence, and the specific productions of engineering at length take their place 
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within the ordered structure of theoretical knowledge. The reciprocal rela- 
tionship between theoretical and engineering science is a fact to which the 
whole history of scientific endeavor attests. 

Another important aspect of mutual support is seen in the dependence 
Of theoretical science on engineering developments for improvements in 
methods and technique. Production of stable behavior functions, such as the 
lever-pressing behavior of white rats, or the production of experimental neu- 
rOsis in animals, provides base lines to which systematic behavioral variations 
can be meaningfully referred. 

Technology is jointly supported by both approaches to science. Within 
the psychological sciences, for instance, applied psychology—the application 
Of scientific laws to particular behavioral situations of practical interest— 
forms one base. The fact that the laws of behavior are largely unknown in no 
way alters the relationship they would bear to a mature psychotechnology. 
The second base is built from the systematized and generalized findings of the 
behavioral engincer. Again, the fact that the contributions of engineering psy- 
chology are relatively scant merely serves to emphasize the need to open up 
this channel of psychotechnological knowledge. 

Although it is a fact of vital importance that applications of abstract 
relationships do emerge as by-products from studies whose immediate purpose 
is to understand, it is nonetheless true that theoretical science is itself appli- 
cation-neutral. In the same way, it must not be overlooked that invaluable 
suggestions for theoretical relationships that “peel off" as bonuses from engi- 
neering efforts are, from a methodological point of view, incidental to the 
Primary purpose of control. 

Most important of all, both research approaches have a common fore- 
bear in exploratory studies whose purpose is the very general one of identi- 
fying and estimating the range of variation of empirical variables. In their 
Present stage of advancement, the great bulk of effort in the behavioral sci- 
ences would appear to fall under the category of variable plotting, which is 
the indispensable precursor of any methodologically designed research. The 
Surprising robustness of exploratory research under the stress of informal and 
intuitive procedures is a fact of greater importance to the psychology of inven- 
tion than to the formal methodology of corroboration. Use of the incom- 
Pletely controlled *quasi-experiment" as an expedient means of teasing out an 
effect from a tangle of factors often yields invaluable insights into funda- 
Mental parameters. Likewise, when supplemented with passive statistical con- 
trol techniques, procedures which are primarily those of the engineer, such as 
those of partial correlation or of conventional factor analysis, may be usefully 
adapted to the derived purpose of elucidating relationships. However, the 
extent to which the methods of either approach are interchangeable with the 
Other may not be merely assumed. The fact that, in practice, scientists com- 
Monly “get away with” ignoring and violating methodological canons without 
Prejudicing the usefulness of results must be interpreted with great caution, 
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It is still true that the researcher who uses theory implicitly and as an after- 
thought, to "verify" his findings, assumes a grave risk. To regard the two 
views as end points on a continuum between theoretical and engineering re- 
search may suffice at some stages of investigation—particularly, as already 
noted, for the informal and approximative purposes of exploratory research. 
However, at other stages, it is difficult to see how either approach can tolerate 
much of any compromise by the other. Indeed, it is the burden of the discus- 
sion that follows to show that certain design options cannot be confused 
without risking serious inefficiency or error. The proper question to be con- 
sidered, then, relates less to the abstract issue of whether theoretical science 
and engineering are complementary or antithetical, than to the extent of 
fungibility of a particular methodological procedure across the two areas, for 
a given rescarch purpose. 

In outline, problems in research design are viewed under the system 
approach as methodological decisions to be determined essentially by the 
broad purpose of the inquiry. Depending on whether the purpose of research 
is that of theoretical or engineering psychology, decisions will differ with 
respect to the choice and formulation of a problem, use of models and hypoth- 
eses, kind of criterion, handling of variability of d 


ata, units of analysis, criteria 
of inference, method of generalization, and kinds 


of emergent action. 
CHOICE AND DEFINITION OF THE PROBLEM 


Psychological research problems are uncertainties in our efforts to under- 
stand or control behavior. The question to be investigated may arise from an 
interest in testing or extending a conceptual model; or from a requirement 
to control—that is, to forecast events for some practical end. Problems in 
theoretical research are generated from a model as implications for testing. 
Since generality of result is the major consideration in this type of inquiry, 4 
parameter or limited group of parameters that promises to have relevance tO 
many different situations is characteristically chosen for study. The choice of 
situations in which the problem is to be investigated will be governed by those 
factors that are implied by some theory as being central to the problem. 
Characteristically, the problem is transposed into a simplified context in the 
interest of controlling irrelevant variation attributable to the particular cir- 
cumstances of the experiment. In discussing Miller's investigations of conflict, 
we noted that although the hypotheses investigated are limited to a few aspects 
of conflict behavior, they are believed applicable to a wide variety of complex 
situations. On the assumption that human behavior is similar to that of lower 
mammalian forms, albino rats are used as subjects to gain simplification and 
control of experimental conditions. 

Problems of engineering psychology are basically different from those of 
theoretical psychology. The question asked will be of a practical sort, on which 
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action within a concrete operational system can be based. How to develop a 
course of training in armor skills which is acceptably better than the con- 
ventional program, such as that developed by SHOCKACTION VI, is a 
representative problem. In principle, the range of problems is limited only to 
that class of situations in which a barrier to some action objective is encoun- 
tered. No assumption is made that the problem has a "unique answer," or 
that it is soluble in closed form. Instead, formulation of a problem of the 
present type requires an exhaustive identification of the facts or variables that 
enter the particular problematic situation. To approximate a reasonably ade- 
quate definition requires that consideration be given to the total operational 
context within which the problem has been generated. If, for example, the 
difficulty is that the performance of the man-tank system fails to achieve a 
required level of performance, and it can further be demonstrated that the 
deficiency cannot be attributed to machine components, the human component 
Would appear to be at fault. Further delineation of the problem within the 
general area of human factors is then required. Selection and placement of 
Personnel, interaction of the machine components with operator and mainte- 
nance personnel, training, management factors (including job structure and 
morale), might all be probed to determine the contribution of each of these 
Subsystems, or some linkage thereof, to degradation of the system output. A 
high degree of generality of the solution is subordinated to the specific needs 
act, the particular situation in all its uniqueness is precisely 
ally seeks to manage. It follows that the conditions 
under which the study is to be conducted should, ideally, be precisely repre- 
Sentative of those which will also obtain under the actual operation of the 
System involved. In SHOCKACTION VI the skills taught were limited to those 
that a task analysis had revealed to be important; and the conditions, to those 
that would be likely to obtain under future training environments. 

The Krogh principle in biology—according to which there is an opti- 
mum species for the solution of any problem—however useful to theoretical 
Tesearch, obviously has little application to the problems of the engineering 
Psychologist. If the risk of immediate applicability of his research is to be 
Minimized, he will enjoy far fewer degrees of freedom in formulating his 
Problem. By the nature of his objective, he is constrained to employ some- 
thing very close to the actual tasks of the military operational system—for 
example, using the tank rangefinder, firing the tank gun, or any of numerous 
Other tasks essential to the armor soldier’s performance of his job. His subject 
Population will be limited to men assigned for armor training, to which his 
findings will be reapplied when integrated into the operational system. How- 
Ever, the sharp limit on the generality of his results will not, within the values 
Of the study, be considered a penalty. In fact, to frame studies of the present 
Sort in terms of a range of factors that attempts to include the goal of gen- 
frality along with that of specific applicability would risk a compromised 
Tesult of little value to either the theoretical scientist or to the engineer. There 
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would appear to be no obvious basis for the commonly held assumption that 
an intermediate range of problems combining both theoretical and practical 
importance can be formulated. 

A methodologically adequate formulation of a research problem of cither 
type requires, before undertaking a study, specification of the set of alterna- 
tives that will be evaluated by the outcome, together with a rational basis for 
acceptance or rejection of each one. Unless the possible actions can be 
explicitly anticipated and evaluated, a rational basis for making design deci- 
sions is lacking. Advance specification of the values that will determine 
acceptance or rejection of the outcome is, within the meaning of methodologi- 
cally designed research, an essential part of defining the problem, and to the 
extent that the basis for such a decision fails to antedate the actual collection 


of data, the research problem, whether theoretical or practical, remains only 
partially formulated. 


THE USE OF ANALOGY 


The explicit role of analogy in scientific procedure is an important char- 
acteristic of the system approach. The question of whether a concept is "truc" 
is transposed to mean under what conditions and with what degree of accu- 
racy an analogy can be usefully employed for a given purpose. The main point 
about analogies is that they provide ways of representation which imply rela- 
tionships among observations. 

A fundamental distinction between types of analogies is made according 
to whether they are to serve the Purpose of subsuming a matrix of observed 
events under an explicitly stated set of conceptual principles, or to provide 4 
basis for controlling the operation of a natural system. The much misused 
term “model” is reserved to denote a set of conceptual relationships whose 
degree of correspondence with relevant observations is demonstrable. The 
methodological status of a model is that of a corroborative or “yerificatory” 
instrument. Miller’s model of conflict behavior is a case in point. His formu- 
lation hinges, not on the assumption that the intervening variables describe 
the actual nature of the organism, but rather on the assumption that theif 
input-output relationships are isomorphic with what is observed. It can thus 
be considered that the organism behaves as if it were constructed according 
to the principles of the model. In general, providing a model satisfies the CO" 
dition of logical consistency, its validity will be decided by the consequences 
of the directed research action which it prescribes, 

The engineering counterpart of the model of theoretical science we shall 
here designate as a schema. The methodological status of schemata is more 
that of a pedagogical device, since they function not to corroborate but t° 
suggest a basis for constructing: a recipe or formula for effecting, or for fore- 
casting, a given end. The engincer’s requirement is met if a criterial state © 
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affairs can be generated—that is, produced (material) or forecast (informa- 
tion) —from another set of variables. Formulas, as here defined, specify in 
the basic equation, Output — (f) Input, what the degree of relationship is. 
Such restriction of the formula to a summarizing statement of the relation 
between input and output is recognized as the engineer's "black box," which 
translates directly into the engineering psychologist's *empty organism." 


The Model of Theoretical Research 


The model of basic research, which is essentially a symbolic idealization 
of observations, subserves a number of closely related functions. 

First, the model guides abstraction from a complex matrix of observa- 
tions, Without some constraints, observations would be illimitable and hap- 
hazard. It is a paradox of theoretical science that "in order to look, the scien- 
list must know what he is looking for." Miller's formulation of conflict 
behavior is, consciously, highly selective and limited to a very few aspects of 
à virtual infinitude of events and possible relationships. On the basis of an 
explicitly formulated model, the problem of what to observe is reduced to 
manageable terms. 

A second function of the model is to translate observed variables that are 
guessed to be germane, from empirical (phenotypic) to conceptual (geno- 
typic) dimensions. These model dimensions, or parameters, represent further 
guesses about variables that can be grouped together on the basis of their 
functional unity, and not merely on the basis of superficial similarity in con- 
lent. The present point is made plain in Miller's studies in the distinction 
between systematic and empirical variables. For example, hours of food de- 
Privation ‘as a generic term includes many empirically different variables, 
Such as the feeding schedule, differences in body weight, and the proportion 
Of various elements in the animals’ diet. Unless we risk the assumption that 
the diverse factors can be consolidated under a single parameter, their mean- 


Ing becomes trivial. 

Third, to qualify as a useful model, t : : 
degree articulated. Ideally, the complete set of functional relationships be- 
tween the several dimensions of the modcl are explicitly specified. Relatively 
few models that are complete in the present sense can be found in contem- 
Porary psychology. In Miller's model of conflict behavior, a number of func- 
lional relationships between the terms of the model are approximated, 
although, as he is careful to note, partially and qualitatively. For heuristic 
Purposes, Hull (1943) pushed his model of habit-strength farther in the direc- 
tion of quantitative articulation, though at the cost of premature precision. 

Fourth, the model must, for certain purposes, permit of being treated 
as a closed system of parameters. The serious difficulties entailed in specifying 
an exhaustive set of behavioral parameters,even for a limited range of phe- 
Nomena, are well known. As a matter of expediency, it is usually necessary 


he parameters must be in some 
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to close the system by invoking the ceteris paribus disclaimer, which assumes 
that other factors are known and, within broad limits, evaluated. The condi- 
tions required for the use of this assumption in psychological research are 
in large part not met, with the consequence that the extent to which error may 
be introduced cannot be formally estimated. Nonetheless, the principle of 
rigorous deduction of hypotheses as implications from a model must remain 
uncompromised. Miller lists a number of such rigorously derived deductions. 
given in simple graphic form. A typical example is that increasing the strength 
of hunger should cause subjects to approach nearer to the goal. 

Finally, in our present consideration of the functions of the model, it 
should supply guidance for inference from its conceptualized dimensions to 
the measures of behavior that are actually observed. A model must have impli- 
cations for the kinds and properties of measures essential to its corroboration. 
In testing his deductions, Miller formulates a number of partial definitions 
relating the terms of his model to specific experimental conditions. For exam- 
ple, it is assumed that greater strengths of electric shock produce greater 
strength of fear drive, and that strength of response is positively related to 
initial speed of running. He is careful to qualify his definitions by calling them 
partial, because their theoretical linkage is not completely worked out. To the 
degree that behavioral indicants remain intuitive, and hence not explicitly 


related to their model, they fall short of qualifying as completely valid cri- 
teria for corroborating it. 


The Forecast Formula in Engineering Research 


In line with its difference in purpose, the forecast formula of the engi- 
neering psychologist differs from the theoretical psychologist’s model in sev- 
eral ways. Primarily, the symmetrical relationship between prediction and 
explanation which is characteristic of an established model has no parallel in 
à forecast formula. Prediction refers to the successful deduction of a rela- 
tionship between independent and dependent variables in advance of observa- 
tion. The correlative term explanation means understanding a relationship 
after it has been observed, by subsuming it under a principle given by the 
model. Forecasting is here defined as indicating in advance the extent tO 
which a state of affairs can be anticipated by means of some set of inde- 
pendently selected variables. The associated term interpretation means the 
specification of a relationship under which an already observed effect may be 
subsumable. Forecasting requires no statement of the logical principle in terms 
of which the forecast is warranted; no rigorous basis for including the ingre- 
dients that enter the forecaster term need be given. In SHOCKACTION Vl. 
the factors comprising the treatment include elements of empirical generaliza- 
tion, common sense, and hunch. The criterial variable is also limited only by 
the requirements of the practice] problem, Or, more accurately, by the re- 
searchers’ guesses about what constitutes an acceptable representation of the 
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state of affairs to be forecast. Again, the proficiency tests which serve as the 
immediate criterion are judged relevant to the combat criterion on the basis 
of expert judgment of researchers and armor officers. The model of theoretical 
rescarch performs the functions of generating highly explicit hypotheses about 
the properties of independent and dependent variables as well as about their 
form of relationship. But, for the purpose of forecasting, there is no require- 
ment to give explicit statement of the basis that may underlie the forecaster 
and criterial terms, or the relationship between them. 


Use of Schemata in Engineering Research 


By contrast with the model of theoretical research, the forecast formula 
offers no rigorous basis for abstracting from the flux of events certain ones 
that "count" as observations. The criterial situation we seck to forecast is, in 
concept, a total interaction of all the variables that comprise it. In SHOCK- 
ACTION VI, the proficiency tests, if they are to reflect the criterion of combat 
efficiency, require in principle no less than a complete photographic repre- 
sentation of everything that armor troops do that contributes to successful and 
unsuccessful combat. Because of the impracticability of meeting such an 
extravagant requirement in practice, approximations must be resorted to in 
Order to simplify the criterial complex and render it manageable. 

The indispensable role of schemata in criterion construction is perhaps 
most readily apparent in research studies which are dependent on some type 
Of representation of the criterial state of affairs because of its cost, inacces- 
Sibility, or nonavailability. Use of simulators as surrogates for actual per- 
formance of systems is essential, for example in complex air-defense systems, 
Where evaluation would otherwise become possible only under actual attack 
(see, for example, Chapter 13). In certain military man-missile systems, 
Schemata serve the invaluable role of making it possible to evaluate per- 
formance prior to the systems’ coming into actual being. The mock-up of the 
design engineer, the wind tunnel of the aeronautical engineer, or the war 
Baming of the military forces provide further instances of the usefulness of 
Schemata in the construction of criteria. 

The variables of engineering research are held close to the level of 
everyday observation in contrast to the highly conceptualized dimensions of 
theoretical rescarch. In SHOCKACTION VI, it would be pointless to render 
into theoretical terms the specific skills and knowledges required to perform 
the armor soldier's task. This is so because behavior as observed directly and 
described in the categories of “common sense” has more relevance to the 
Practical problem in hand than would a higher order of abstractions. 

A further difference between the engineering and theoretical approaches 
Telates to the absence of any formal requirement to specify causal relation- 
Ships, or mechanisms, between the forecaster and criterial terms of the fore- 
Cast formula. Once more, it is not a primary function of the forecast formula 
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to make possible attribution of a dependent variation to a particular manipu- 
lation of an independent factor. Rather, the requirements are those of any 
efficient. psychometric instrument—unambiguous identification of predictor 
and criterial variables, together with a demonstrated degree of association, 
under specifiable conditions, between the two terms. To the extent that Rem 
mata aid the engineering psychologist in building his formula to meet these 
three specifications, and not because they provide any formal basis for infer- 
ence, they may be usefully employed. 

Lack of a requirement for rigorous grounds of inference does not offer 
a complete basis for distinguishing between schemata and models. Forecasts 
are often aided by schemata which are both explicit and highly rigorous. Both 
function as analogies to represent specified sets of observations, but beyond 
this similarity, they are fundamentally different: unlike the model, the fore- 
cast formula is not corroborated, in any formal sense, by generated outcomes. 
Only by the gross evidence lent by the success or lack of success of the 


global product of the particular study is a schema "checked" for its corre- 
spondence with relevant observations. 


© VALIDITY VERSUS FIDELITY 


The term validity refers to the degree of correspondence of a model to 
germane observations. To the extent that it can be demonstrated to be homol- 
ogous with observation, a model is valid for the purpose of establishing 
scientific laws. To illustrate the point, we turn again to Miller’s theory of 
conflict behavior, which he attempts to validate by checking the extent to 
which the logical implications from the model are borne out by a program 
of carefully arranged experimental observations. 

To distinguish validity as a concept appropriately applied only to theo- 
retical research from its counterpart in engineering, we have somewhat arbi- 
trarily labeled the latter fidelity. The sufficient condition which defines fidelity 
is that a given variable permit us to forecast a second, criterial variable to the 
extent required by the problem in hand. In SHOCKACTION VI, the devel- 
oped program was judged sufficiently likely to forecast performance under 
future operational conditions to warrant its adoption by the Army. 

It is relevant in the context of the present discussion to make a further 
distinction between an indicant, as employed in theoretical research, and a 
criterion, which is its engincering-research counterpart. Although the two con- 
cepts are often interchanged in practice, their defining functions are by nO 
means identical. The term indicant focuses attention on the role of the de- 
pendent variable as a basis for inferring the values of the model parameters. 
Selection of the indicant is prescribed by the theoretical model to reflect 
changes resulting from a given independent variation in experimental condi- 
tions. Variation of indicants is determined by the values assigned to the inde- 
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pendent variables. without restriction of range. A criterion is the wanted 
output which a to-be-developed input will yield; its value will be assigned by 
the set of practical considerations that have generated the problem. Both in 
respect to the logic and sequence of research, the criterion is prior to the 
forecaster term which it governs. In our reference study, SHOCKACTION VI, 
it is seen that the components of the engineered course of training must, above 
all, meet the standard of judged combat relevance. In brief, the engineer 
reverses the direction of the cause-to-eflect reasoning of the theoretical scien- 
tist by "working back" from a specified output to a set of generating 


conditions. 


The Operational Definition 


Operational definition is the term Bridgman (1927) has given to link- 
ages between the antecedent conditions and deduced consequences of a model 
and relevant observations. As he is careful to point out, definitions of the 
present type can be formulated only for highly idealized situations, to embrace 
à set of phenomena whose interrelationships are already known or accurately 
Surmised. Miller's partial definitions of strength of fear drive and response 
Strength, previously considered, are probably as good approximations to 
Operational definitions as the present state of psychological theorizing affords. 

Two serious misconceptions about the nature and purpose of operational 
definitions have plagued much psychological research. The first is the unten- 
able position that a concept of the present sort can be defined as the sum of 
empirical operations required for its demonstration. A familiar example is 
the attempt to define hunger motivation exclusively in terms of the laboratory 
Operations involved in withholding food from an animal for a specified inter- 
Val of time. Or, to take another instance, unit morale in the Army is often 
defined “operationally” as the frequency with which members of a military 
Organization go AWOL, "ride the sick-book,” or otherwise demonstrate be- 
havior that is regarded as undesirable. Without question, the property of 
Objectivity which these definitions emphasize is an essential requirement of 
any scientific research. However, more than a lack of ambiguity of terms is 
required to perform the function of bridging model and observation. To satisfy 
the additional requirement, the objectively defined measures must be explicitly 
Telated to the model to which they stand as indicants. Operational definitions, 
Properly understood, are a tool of theoretical research. Their purpose is to 
make concepts unambiguous, not to make it possible to theorize without con- 
cepts. Restriction of definition of psychological concepts to empirical variables 
is a common confusion that misses the essential point of the theoretical 


Approach to science. à ; 3 , 
of the one just considered, is committed 


A. second error, the converse red 
When a requirement for an operationally defined concept is imposed on an 
t advocacy of "construct validity" appears to 


*?ngineering problem. The curren 
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offer a case in point (Cronbach and Mechi, 1955). Interposition of a con- 
struct between the forecaster and criterial terms in an enginecring investiga- 
tion is not merely superfluous. but can scarcely avoid resulting in inefficiency 
through sacrificing direct relevance of the forecaster term to the criterion. 
For the purpose of forecasting, empirically defined terms are sufficient. If the 
goal of avoiding the limited generality of a particular testing instrument is 
sought, a theoretical reorientation to the problem is called for. Only by means 
of hypothesis-testing research can the Parameters that underlie empirical rela- 
tionships and the conditions of their variation be rigorously demonstrated. 


THE USE OF HYPOTHESIS 


In theorctical research investigations, hypotheses are generated from à 
model as logical implications to be tested by experiment. In form, the hypoth- 
esis specifies a relationship between a dependent and an independent variable. 
Testing of such a relationship consists in demonstrating its tenability under 
appropriate rules of inference. The controlled experiment, which is essen- 
tially a hypothesis test, permits the attribution of variation in a dependent 
variable to an introduced manipulation of an independent variable. The goal 
of hypothesis testing is causal analysis. The characteristics of the hypothesis 
in the present sense are readily observable in the reference experiment by 
Miller. 

The “hypotheses” of engineering psychology contrast with those of theo- 
retical research in origin, in form, and in purpose. No a priori constraints 
are placed on the selection of schematic relationships that may be employed 
in connection with the forecast formula; they may be entirely unsupported by 
structural knowledge. The alternative to be tested consists of two possible 
courses of action, and the question to be answered is which one better fore- 
casts the designated criterion, The purpose of the engineering study, as We 
have already noted, is not that of attributing a dependent variation to a par- 
ticular independent variation, but rather of demonstrating that a desired out- 
put can be achieved by a specifiable set of input conditions. 

SHOCKACTION VI, which was designed to produce an improved 
training program, employed a complex treatment, the components of which 
were drawn from various sources, including Psychological theory, rules of 
thumb, and training lore. It was not important for the objectives of the study 
to state the basis on which it was assumed that such ingredients as inclusion 
of knowledge of results, trainee participation, or whole-to-part learning might 
improve proficiency. Further, it is apparent that the aggregation of variables 
in the design of the experiment does not permit any single factor or group of 
factors comprising the treatment to be isolated as causal. The degree of 
aggregation of variables is determined by the requirement for their manage- 
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ability within the operational system concerned. For the purpose of studies 
of the present type, it would be inefficient to insist on conformity with the 
ideal of the analytic experiment in maintaining identity, except for a unitary 
independent variation, between the experimental and control groups. What is 
demonstrated by the outcome is an association between the global interaction 
represented by the treatment, and the total effect. Nor is any further inter- 
pretation needed to decide between the developed and standard programs of 
training. In general, the engineering study provides a basis for choosing 
between the two courses of action by successfully demonstrating that certain 
values of one total set of conditions yield an output greater than that yielded 
by different values, or factors. The distinctions noted between the two kinds 
of experiments might be better served if that of theoretical research were con- 
sistently designated as a hypothesis-testing experiment, in contrast to the 
demonstration-experiment of engineering psychology. 


Control versus Representation of Variability 

retical scientist and the engineer are 
Sharply reflected in their different approaches to the problem of variability of 
Observations. To the "pure" rescarcher, error means error of observation—that 
is, failure to control factors not included within the parameters of the model. It 
follows that unless observations can be related to a model, a basis for determin- 
ing factors to be controlled is lacking. Because determination of laws of behavior 
depends on elimination of error introduced by accidents of observation, 
simplification of conditions is necessary. In the interest of controllability. 
Miller, as we have noted, limited his experimental conditions to those of a 
Very simple kind. For the purpose of theoretical research, the sophomore 
often suffices as a subject, the unnaturally restricted atmosphere of the labora- 
tory as a setting, the tachistoscopically presented nonsense syllables as the 
independent variation, or the manual pressing of a key as the isolated measure 


of response. 
To the engineering researc 


The different purposes of the theo 


her, error refers mainly to a failure to com- 
pletely represent essential criterial variables among the factors included in the 
forcaster term. For the purpose of forecasting, the particular variabilities of 
Observations in the criterial situation must be accepted as given. If, in 
SHOCKACTION VI, men of low aptitude will actually be included among 
the population who will undergo training under conditions of operation, such 
Subjects cannot be excluded from the experiment; if the tank nomenclature 
to be mastered consists of complex, meaningful verbal symbols, a study that 
employed nonsense syllables would not be efficiently designed to maximize 
Successful prediction; if driving skills must be performed under the noise and 
Stress of battle, these factors may not be omitted from the developed train- 
ing without loss of efficiency. The general requirement for this type of study 
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is that the populations of subjects, treatments, environmental factors, and 
performances which comprise the complex system of interest to the practi- 
tioner be represented as closely as possible in the forecaster term. 

Allowing for the important exception of varying several model param- 
eters simultaneously to determine their interactions, classical design attempts 
to eliminate or hold at a constant value all variables other than the experi- 
mental variable. In principle, the “rule of one variable" obtains in research 
designed to make attribution of effect to cause possible. Thus, the quest for 
uncontaminated, unidimensional factors which theoretical research demands 
would appear to be best met by univariate research designs. 


Univariate versus Multivariate Design 


Confusion between the two ways of dealing with variability has led to 
the erroneous view that the use of experimental controls in an observation 
necessarily decreases the generality of the obtained findings—an argument 
frequently adduced in favor of multivariate design. R. A. Fisher (1951) 
makes the point that the exact standardization of experimental conditions 
always entails the disadvantage of yielding. information that is restricted to 
the narrow set of conditions permitted by the standardization. He argues 
further that standardization weakens rather than strengthens the grounds for 
inferring a like result since, in practice, conditions are never invariant. From 
the point of view of the agricultural engincer, which Fisher represents, thc 
statement is valid. However, a wide inductive basis for application of results 
is of less interest to the theorctical scientist than their precision under highly 
restricted conditions. Systematic variation of conditions for the theorist serves 
the purpose of delineating the scope of a model, rather than that of demon- 
strating its applicability over the broadest possible range of conditions. Com- 
bining several conditions within a study of multivariate design more often 


than not denies a basis for inferring that a model holds under any one of the 
particular conditions. 


Representative Design 


Brunswik (1956) carries the case for representation of variability to its 
logical extreme, arguing for abandonment of systematic control in favor of 


registration of the totality of variables by passive observational procedures. 


His position is prescribed by a fundamental interest in biological adaptation— 
that is, in the functioning of organic Systems with reference to their goal of 
achieving behavior states which are adapted to "representative" environments. 
So stated, the problem that interests Brunswik is seen to be cast in the form of 
an engineering task of a special kind: how have the processes of natural selec- 
tion designed and developed organic systems to accomplish the goal of sur- 
vival under the complex ecological conditions to which they must adapt? 
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This problem, although essentially one in engineering, differs from those of a 
more conventional type in two ways: first, in the respect that natural process 
stands in surrogate for the engineer, who commonly controls the develop- 
ment of the system himself; and, second, in the respect that a methodology 
primarily designed for the purpose of control is being employed to facilitate 
understanding. Approaching problems of behavior from the standpoint of 
their adaptive function has proved its great utility to theoretical science, with- 
out, however, answering the fundamental question of the laws under which 
behavioral processes mediate adaptation. Our consideration of the issue makes 
it clear that representative and systematic design are not to be viewed as 
completely antithetical, but in certain ways as complementary procedures for 
dealing with the problem of variability of data. 

A rule for determining the extent to which a "realistic" environment may 
be required in a particular research investigation emerges from the methodo- 
logical distinction between representative and systematic design. It is inherent 
in controlled situations that they are artificial; no demand is imposed on theo- 
retical research investigations that they be otherwise. The special arrange- 
ments of the laboratory are wholly intended to reduce the accidental variation 
of "real-life" situations in order to permit observation of an uncontaminated 
relationship between independent and dependent variable. Conditions of 
Observation that permit a welter of unidentified variables having unknown 
consequences to play without restraint seldom illuminate model relationships. 
Equally, it is essential in the approach of the behavior engineer that the 
inductive leap which generalizes from the forecaster term to the criterial situ- 
ation be held at a minimum. Since the criterion is a particular value of a state 
of affairs in the world of operations, a good deal of realism is called for. 
Inevitably, the cost for such close relevance to the complex circumstances of 
Practical life is paid in the limited generality of results obtained under natu- 
ralistic conditions of observation. The extent to which realism may be effec- 
tively traded for control can be seen to depend on the nature and purpose of 


the particular research. 


Large versus Small Numbers of Observations 

osely related to the representation-control 
f the number of observations on which a 
tely based. Although a definitive answer 
r of considerable complexity, the prin- 


ciples outlined above provide some general guidance. In studies whose pur- 
Pose requires that the total variation of the population be represented, use of 
Small samples entails a disproportionate risk of introducing error which can 
result in erroneous generalization of an extreme value to the underlying 
Population. If, on the other hand, the purpose of the research defines an 
interest in some few factors that are known, or guessed to be the essential 


A further design option cl 
dichotomy raises the question of. 
Particular research can be appropria 
to this problem is obviously a matte 
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determinants of the variation, then, even in the face of the risk, the greatest 
likelihood is that effects of the primary determinants rather than those of 
secondary factors will be observed (Sidman, 1960). Multiplication of in- 
stances, as an alternative to increasing experimental control, would, under the 
circumstances, tend to increase the heterogeneity of data without yielding a 
basis for assigning the additional variation to particular determinants. In brief, 
it can be seen that reliance on large numbers of observations generally belongs 
with the representative designs of engineering psychology, and that use of 
small samples is associated with the systematic designs of theoretical research. 


DEFINITION OF UNITS AND LEVEL OF ANALYSIS 


An important set of methodological options that arises in any research 
effort is concerned with the delimitation of units which are isolable from the 
complex interplay of observed events. Classically, this problem has been 
handled by the assumption of ultimate and "irreducible" units, as for exam- 
ple, the sensation of Titchener's Structuralism, or the reflex of Watsonian 
Behaviorism. More recent theorists have tended to make some increased con- 
cession to a relative concept which assumes a hierarchy of levels of analysis. 
as Tolman (1932) and Hull (1943) do in their concept of molar behavior. 
Miller, who works largely, though not exclusively, at the behavioral level, 
recognizes that the most profitable level of analysis within an area of inves- 
tigation depends on the phenomena of interest, and on the stage of develop- 
ment of the relevant disciplines. The system orientation goes a step farther 
in the degree of explicitness with which it assumes that constituent elements 
are not "given in nature" but are to be defined under the objective of an 
inquiry. 

For the purpose of theoretical research, definition of units is constrained 
and guided by the research model. When a given unit has proved successful 
in demonstrating generalizable behavior uniformities, under specified condi- 
tions, it may be considered valid for its purpose. As Skinner (1938) has 
noted, behavioral units are defined at levels of specification marked by the 
orderliness of dynamic changes. Operant behavior offers an almost unique 
example of a behavioral unit that has been demonstrated to enter into a large 
variety of lawful relationships. 

If, on the other hand, the objective of a research task is to demonstrate 
that a given treatment will improve the efficiency of an operational system, nO 
formal requirement for analysis of the treatment, or of its effects, is imposed. 
An attempt to break the complex into component factors would be inefficient, 
since it is the totality of their interaction that is of inerest. Thus, in SHOCK- 
ACTION VI, the individual contribution to the several aspects of performance 
made by content, methods of teaching, and motivational devices are of less 
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importance than that of the total interaction of these factors operating jointly 
on gross performance. In the present connection, Brunswik's concept of the 
"ecological package" provides a cogent argument for the need to define the 
unit as the total situation to which the organism must adapt. Aggregation of 
factors is an invaluable engineering tool; if the total complex of factors can 
be feasibly identified and administered with a gain in efficiency in the system 
involved, it is a valid unit for the engineer's purpose. 


STATISTICAL INFERENCE 


[s] 


The problem of statistical inference is concerned with the acceptability 
of the results of observation for a designated purpose in face of an estimated 
likelihood of error. Criteria of inference will differ depending on the purpose 
of the investigator: whether to justify inclusion of the outcome of a hypothe- 
sis test within a comprehensive postulate structure, or to demonstrate that one 
course of action offers a more efficient solution to an administrative problem 
than another. 

In theoretical research, our chief concern is with the acceptability of a 
set of observations for testing an implication derived from a model. With cer- 
tain exceptions that fall beyond the scope of the present discussion, model 
relationships are, by virtue of their conceptual status, deterministic rather 
than probabilistic in nature. Therefore, the probabilities assigned to a 
hypothesis are restricted to the two values, one or Zero; the hypothesized 
relationship is accepted as “true” or rejected as “not true.” The corrobora- 
tion of a hypothesis is thus an all-or-none affair based on the decision that 
its acceptance is more useful than its negation, for the purpose of building a 
conceptual system. Because the decision to accept or to reject does not admit 
of gradations, the salient question becomes one of whether or not It will be 
considered that an observed change is attributable to a specified independent 
variation, rather than the question of the likelihood of its occurrence or of 
the magnitude of its effect. The question to which an answer Is sought is 
whether the conceptual relationship holds under a specified set of conditions. 
Apart from the matter of the degree of replicability of findings which the 
theoretical scientist finds essential to experimental study, it would be difficult 
to assign a meaning to such a statement as, for example, ‘the probability that 
this relationship implied by Millers model of conflict is true Is 95. As 
already noted, 10 deduced implications were accepted by his series of experi- 
ments, and 1 rejected. This impressive record has an important bearing on the 
validity of the model, although the question of precise statistical significance 
Poses a problem that has received no completely acceptable answer: 

Conventionally, a finding is considered significant when it can be sub- 
sumed under a rule that would minimize.the risk of accepting, within the 
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scientific structure, relationships that might more usefully be considered to 
result from random factors. To accomplish this objective, the researcher sets 
up a statistical counterhypothesis (null hypothesis) which assumes that an 
observed variation in the dependent variable is not the result of the experi- 
mental manipulation, but rather that of chance variations in the experimental 
conditions. If the counterhypothesis is then rejected, one possible set of 
determinants of the result —chance factors—is considered to be negated, and 
the alternate hypothesis, as one among the remaining set of possibilities, is 
regarded as corroborated. In much theoretical scientific practice an extreme 
convention is followed in determining the acceptability of the alternate hy- 
pothesis—the requirement that the null hypothesis be rejected at the .05 
(.01) level. Insistence on what is, in effect, an absolute criterion of signifi- 
cance is usually rationalized on the grounds of the serious consequences 
resulting from acceptance of erroneous findings into the superstructure of 
scientific knowledge. Although application of the concept of probability to 
theoretical relationships appears to have no clear bearing on their corre- 
spondence to a given set of observed data, statistical inference procedures are 
essential to the technical purpose of insuring sufficient stability of an observa- 
tion to make its investigation possible. 

The criteria of acceptability of findings appropriate for the purpose of 
the engineer differ in several important respects from those outlined imme- 
diately above. The alternatives tested by the demonstration experiment are 
not model relationships, but are, instead, possible courses of action in the 
world of operations. The universes of Observations on which inductive infer- 
ences of the present type rest can never be exhaustively sampled. Therefore, 
the probability values which outcomes may assume will vary continuously 
between the limits of zero and one, without attaining either value. Moreover, 
it seldom makes sense to employ the value of chance as the baseline from 
which a practical outcome must differ in order to achieve significance. The 
alternative to be compared with the experimental treatment is virtually never 
based on complete ignorance. The basis of comparison will, hence, be some 
alternative procedure that meets the requirement of action with some finite 
degree of efficiency. A standard often employed for this purpose is the exist- 
ing action procedure; for example, SHOCKACTION VI used the conventional 
course of armor training. In general, the question of acceptability of findings 
in engineering studies takes the following form: at a specified level of risk. 
should action alternative A, which has been demonstrated to yield a cri- 
terion score higher by a given increment, be adopted over action alternative 
B, for situations that fall within the class of the one investigated? 

The practical issue is usually whether the degree of likelihood of repro- 
ducibility under operational conditions, taken together with the increment of 
improvement in terms of gains and costs, are sufficient to justify one line of 
administrative action over another. In making such judgments, Type II statis- 
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tical errors, which weigh the cost of erroneously rejecting useful courses of 
action, are appropriately given more consideration than they are in theoretical 
research. It is apparent, for example, in the case of the mentally incurable, 
that a small increment of improvement observed in response to administra- 
tion of a new drug would often be sufficient basis for more extended use— 
even when the risk that its effects would not be duplicated in actual practice 
may be considerable. On the contrary, rejection of the treatment under such 
conditions should be based on the most cogent counterevidence—damaging 
side effects, death of the patient, or other equally undesirable consequences. 
Particularly in the case of military decisions, the need for improved courses of 
action may often be great enough to warrant acceptance of a small advantage 
together with grave risk of failure. Saving two weeks’ time through the use 
of the training program developed by SHOCKACTION VI could, almost 
regardless of other costs, represent a critical gain under the necessity for 
rapid military mobilization. Criteria of acceptable inference that minimize the 
likelihood of Type Il errors are seldom appropriate for practical problems. 
Indeed, the methods of statistical hypothesis-testing generally appear to be 
better suited to the requirements of the theorctical researcher. The methods of 
statistical estimation, which permit statement of the limits within which a 
difference may range, are more directly applicable to the problems of the 
engineering psychologist. The practitioner is usually more interested in the 
estimation question of how much difference, than in the hypothesis-testing 
question of whether there is any difference. 

Confusion between statistical and practical significance in the research 
literature takes a variety of forms. One of the most common errors of the 
present type is the view, usually adamantly held, that the stringent statistical 
criterion of theoretical research alone defines "real" significance as contrasted 
With some second-class or makeshift variety. Scarcely less incongruous is the 
growing practice of assigning two significance values, one statistical, the other 
practical, to the same research outcome. Not uncommonly, small differences 
measured between groups, when statistical significance is mainly accountable 
in terms of a large number of observations, are of trivial consequence to the 
administrator for whose purposes significance must be measured in dollars 
or in other comparable gains and costs. Under other circumstances, a small 
difference that falls considerably below the criterion of conventional statistical 


significance may represent a finding of extreme practical value. 


E PRINCIPLE VERSUS POPULATION GENERALITY 


[tj] 


Up to now our consideration of generality has been limited to the ques- 
he quantitative incidence of certain 


tion of reliability—that is, whether t i ; 
characteristics noted in a sample of observations will also hold of its parent 
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population. Another meaning of generality, however, deals with the exten- 
sibility of findings beyond the set of conditions under which they have been 
demonstrated to hold. The following are examples of the present kind of 
question: can Miller's preliminary model of conflict behavior be extended to 
subsume the complex phenomena of displaced aggression; or, will the 
SHOCKACTION VI training, which has been developed on a sample of 
trainees limited in range of age and geographic origin, continue to yield supe- 
rior scores when applied to a new batch of trainees who differ in these char- 
acteristics? The questions suggest two distinguishable bases of generalization. 
One is of theoretical importance, and rests on the assumption that an invari- 
able relationship applies in a particular case. The other, of practical impor- 
tance, is based on the premise that a given sample is representative of its 
universe. 


Population inferences are grounded in the logic of classification. Sharp 
limitations are imposed on generalization of the present type by difficulties 
encountered in specifying relationships between classes. An important example 
of this weakness is seen in the field of psychometric testing, where the rela- 
tionships among a large number of "blind," ad hoc, testing instruments have 
long continued to remain unknown. The same kind of limitation holds of 
studies like SHOCKACTION VI, which yield no formal basis for inferring 
that the improvement in efficiency observed in the experimental situation 
would continue to hold if conditions were changed in any way. Some incre- 
ment of generality may, of course, be attained by empirically demonstrating 
that results are not impaired by a particular alteration in the attributes that 
define the class of such studies, But the number of possible attributes to be 
eliminated in this way is, in any practical situation, usually so formidable as 
to preclude any high degree of generality based on content, As a matter of 
practice, informal guidance is often provided by common sense, or by accu- 
mulated empirical findings with Tespect to particular variables or their 
magnitudes, that can be altered without compromising the result. As a con- 
sequence of the difficulties inherent in systematizing the accumulated findings 
of individual engineering studies, a science of Psychotechnology, which relies 
heavily on the integration of such outcomes, exists mainly as i program for 
future effort. Engineering studies are neither designed nor undertaken pri- 
marily to yield an integrated body of principles. Systematization of findings 
derived from studies undertaken essentially to cope with practical problems; 
in which many different unanalyzed complexes have been permitted to vary 
in unknown ways, poses a task which yields to attack only grudgingly. 

Principle generality bases its inferences on the logic of relations. Group- 
ings of observations into classes according to their attributes are replaced by 
dimensions that permit observations, withi ^i 


: x n defined limits, to assume con- 
tinuous values, and that can be felated to other dimensions prescribed by 4 
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particular model. In the reference study of conflict behavior, Miller's model, 
for example, relates the dimensions of approach, avoidance, motivation, and 
reinforcement, within defined limits. Generalization becomes a matter of 
extending the scope of a limited model to a more complex set of phenomena. 
An effort is made to determine whether the new conditions are of the sort to 
which the model has application. If these antecedent conditions have been 
demonstrated, by means of independent criteria, to obtain, the consequence 
specified by the model must follow. With the preliminary establishment of a 
model relationship, further research efforts will be directed at determining, 
systematically, results yielded by other combinations of the parametric values. 
Miller's subsequent extension of the preliminary conflict model to certain 
phenomena of projective testing illustrates, in a very primitive way, use of 
generalization procedures of the kind under consideration. 


[ ABSTRACT PREDICTIONS VERSUS FORECASTING 


The outcome of a controlled laboratory investigation yields a basis for 
abstract prediction. When the specified conditions under which a generaliza- 
tion has been demonstrated obtain, the result of a change in the independent 
factors can be predicted with precision. Thus, it can be asserted with a high 
degree of confidence that under the pure conditions of the chemical labora- 
tory, water vapor, when contained within a confined volume of air at a given 
temperature and pressure, will be precipitated. Similarly, in the area of human 
factors, numerous studies have demonstrated that, ceteris paribus, acquisition 
of a habit is facilitated by letting the learner know how effective his per- 
formance has been. For the purpose of theoretical science, the goal of an 
investigation is achieved with the statement of a general relationship a the 
present type—that is, a conclusion. For application to a real life pip: n 
however, the closed system of parameters must be opened up to the in i 
ences introduced by the complex environment of a particuiar operational 
system. Thus, to apply the gas laws to a set of actual meteorological ir 
Tequires, in addition to an understanding of the basic relationships invo ved, 
evaluation of the conditions that obtain within the air masses involved. Like- 
wise, in an actual training situation, to assert with confidence that feedback 
of results will improve learning hinges on the possibility of AR and 
evaluating the relevant empirical factors included in the pars o iiie 
Knowledge of such factors as the amount and kind of deem E e ipis 
length of practice sessions, incentives, and rewards an E EE = : 
be required to provide a sound basis for forecasting successful articulation o 


the principle of feedback into the system of training involved. Thus an abstract 
relationship may yield an implication for action, but without a formal estimate 
of the likelihood of its applicability to a particular set of operational condi- 


544 - The System Concept and Methodological Decision 


tions it fails to lend a complete basis for recommending that a o vae a 
course of action be taken. The field test plays a crucial role in ai ai 
gap between abstract predictions based on theoretical relationships, an 
concrete variable interactions of actual environments. an 

In contrast to the indeterminate degree to which abstract predictions 
may apply to a particular set of operational conditions, the relevance or rad 
neering forecasts is direct and immediate. This important difference ste : 
from the fact that the behavioral engineer builds the higher order or 
actions among the situational variables into the forecaster terms of his formu a. 
Thus, in SHOCKACTION VI, the developed training program was tested in 
the context of actual Army operations. The closer the degree of similarity 
between the experimental conditions and those that might be presumed to 
hold later in the criterial situation, the greater the likelihood that the = 
oped program would work when plugged into the Operational system s 
armor training. An engineering forecast, then, provides a direct basis for 
recommended action because the degree of replicability of the effect of the 
developed conditions within the total System environment can be estimated. 
As a practical matter, operational environments are often unstable, and the 
effects of extraneous factors that are unrepresented in the forecaster term 
cannot be taken into account. To cope with such unforeseeable intrusions, the 
engineering psychologist finds it necessary, as a matter of practice, to hedge 
his recommendations by means of ample safety factors. 

Finally, to assert that the outcomes of theoretical research studies are 
stated as conclusions, while those of engineering research are framed as rec- 
ommendations, is true only in a relative sense. The results of either approach 
to research may have implications for both the systems of knowledge and 
action. Both are subject to the reign of the pragmatic principle, but under 
value structures which are different. The engincer’s action must be imme- 
diately tested against those values that inhere in the world of affairs, while 
the action taken by the theoretical scientist must meet that criterion of value 
that is appropriate to building a system of knowledge. 

Whether a stated outcome is a recommendation or 
only be defined relatively to the purposes of action. Viewe 
limited system of science, the conclusions of theoretic 
implicit recommendations for th 
to extend our understanding of 
neer’s recommendations are ac 
have not yet been transformed 
theoretical purposes. From th 


a conclusion can 
d from within the 
al research are, in fact, 
at special kind of action that secks primarily 
nature. From the same perspective, the engi- 
corded the status of information, since they 
into principles of action of the type useful for 


€ more inclusive viewpoint of the world of 
affairs, the engineer’s recommendations Serve as formulas for immediate 
action directed toward controlling the environment; 
theoretical scientist now serve the Purpose of inform 
the practical requirements of action. 


the conclusions of the 
ation to be adapted to 


—John L. Finan . BUE 
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stimulus characteristics, 292 
stimulus control, 296 
system operations, 275 
task supported by, 284 
thinking activities, 287 
training, 294 
varieties, 293 

Job analysis 
training development, 315 
training study, 318 

Job manual, and memory, 326 

JoHNsON, D. M., 51 

JOHNSON, E., 478 

Jones, F. E., 334 

JoRDAN, N., 153 


Kappaur, W. E., 175 
KrasT, W. R., 306 
KENNEDY, J. L., 23, 26, 469 
KENT, A., 110 
KEPNER, C. H., 378, 479 
KiBBEE, J. M., 368 
Kipp, J. S., 52, 71, 173 
Kinalog display, 176 
KiNGsLEY, H. L.,.333 
KINKADE, R. G., 173 
KISTER, J., 110 
Kraus, D. J., 411, 468 
Knowledge 
skill, 318 
specification for training, 315 
Knowledge of results, 375 
individual and team, 376 
and system training, 382 
KRAFT, C. L., 177 
KRENDEL, E. S., 146, 148 
Krogh principle, 527 


Language, computer translation, 110 
LEAKEY, L. S. B 
Learning 
knowledge of results, 375 
management of, 324-338 
relation to job aids, 294 
system, 513 
Legibility, displays, 174 
LICKLIDER, J. C. R., 136-138, 140, 141 
Lighting, ambient, 118 
LiNDQUIST, E. F., 507. 511 
Load constancy, 171 


Index 


Load-sharing, in system task, 71 

Locke, W. N., 111 k 

Logic, computer programming, 109 

Logistics, in task description, 195 

Lons, E. R., 162 d 

Loops, computer programming, 82 

Loucks, R. B., 175 

Loudness, auditory, 137 

LUCHINS, A. S., 164 

Luminance, visual requirement, 133 

LUMsbAINE, A. A., 284, 337, 346, 350, 351, 
352 


McCartny, J., 110 

MACCAaSLIN, E. F., 318, 321, 522, 524 

McCormick, E. J., 396 

McCu.Locu, W. S., 108 

McGeocu, J. A., 48, 333 

Machine(s) 
alteration, and team functioning, 392 
functional characteristics, 233 
procedures, and teams, 388 

MacHoLr, R. E., 146 

Mackie, R. R., 455, 460 

MACKWORTH, J. F., 67 

MackKwonTH, N. H., 67 

MACPHERSON, S. J., 376 

MCcRuER, D. T., 146, 148 

MAHLER, W. R., 373, 378 

MAIER, N. R. F., 328 

Maintenance 
category of task description, 195 
equipment, proficiency, 465 

Man 
comparison with computer, 92-96 
differences from machine, 233-237 
element in control systems, 145-148 
functional characteristics, 233 
role in man-machine system, 36 
system component, 122 

Management subsystem, 18 

Manager selection, 398 

MANDELBAUM, J., 130 

Manpower requirements, and jobs, 241 

Marcu, J. G., 396 

MARTIN, L. B., 352, 353 


Masking 
auditory stimuli, 139 
speech, 139 


MATHERS, B. L., 319, 320 

Matrix prediction, in selection, 253 

Matyas, S. M., 330 

Mayo, E., 397 

Measurement 
attributes, in selection model, 254 41 
proficiency, contamination and bias, 4 
proficiency, defining purposes, 431 
proficiency, definition, 420 


proficiency, random errors, 443 
easures 


“a in selection and classification, 
54 


characteristics, 256 


criteria, in selection and classification, 
263 


—Index 


objective, in personnel measurement, 257 
MEEHL, P. E., 534 
MELTON, A. W., 334 
Memory 
codes, in task analysis, 210 
computers and men, 94 
human, long-term, 40, 45 
in identifying, 48 
in interpreting, 51 
job manuals, 326 
related to system design, 164 
short-term, in human functioning, 45 
task analysis, 208 
MznzprrH, W., 266 
MILLER, E. M., 361, 381 
MILLER, G. A., 140 
MILLER, N. E., 464, 520, 521 


MILLER, R. B., 58, 212, 217, 221, 222, 225, 
284, 374 

MILNER, P. M., 108 

Missile(s) 


guidance, controls design, 180 
operators, training, 327 
Missile system, checking out, 69 
Mission(s) 
cycle, in organization of training, 217 
and personnel subsystem, 239 
segments, 199 
Mock-up(s) 
automobile engine, 348 
cutaway, 347 
nonoperational, 347 
operating, 347 
Model(s) 
human memory. 45 
man, by computers, 107 
prediction, in selection 


Moraan, C. T., 491 

orse code 

coding task, 63 

computer task, 94 
Moss, F. K., 131 
Motion pictures, 346 
Motivation 

crews in system experiment, 512 

trainees, 328 

for training. 329 

otor response 

in task analysis, 214 

training, 225 : 
Motor skills, and identifying, 49 
Muck er, F. A., 374 


Neural structures, simulation, 107 
EWELE. A., 23. 91, 93. 109, 110 

Newit 67 

Nicuors, I. A.. 373, 378 

Nicki as, D. R., 373 

Noise 


and classification, 
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in display design, 174 

effects on human performance, 150-151 
filtering, in task analysis, 207 

masking of auditory stimuli, 139 


Observations, in research, 537 
OLDER, H. J., 461 
Olfaction, 144 
On-the-job training (OJT), 356, 405 
Operation(s) 
analysis of, for training, 314 
category. task description, 195 
hunter-killer system, 97 
integrating training program, 382 
system, job aids, 275 
system testing, 8 
training, 19 
Operation School Burning, 407 
Operational definition, 533 
Operational effectiveness, 36 
relation to function allocation, 154 
Operator, 161 
see also Human operator 
Optimal classification model, 269 
Organism, synthetic, 13 
Organization 
in controls design, 179 
imbedding, in system simulation, 488 
Orientation training. 217 
OrnSTEIN, G. N., 373. 378 
Osburn, H. G., 328 
Oscilloscope patterns, job aids, 297 
OSS Assessment Staff, 436 
OsTERBERG, G., 126 
Output, human, 42 
Output processing, 168 


PALMER, R. R., 306 

Parachute rigging, checklist, 455 

PARKER, J. F., 336 

Pattern construction, 169 

Pattern recognition 
computer programming for, 108 
computers and men, 94 
job aid, 297 

PELTA, E., 512 

Perceiving, 41 
human functioning, 55 
relation to interpreting, 51 

Perceptual constancy, 48 

Perceptual interference, in identifying. 163 

Performance 
air-defense direction center, 502 
average level, in personnel measurement, 

256 
changes resulting from practice, 513 
criteria, from task description, 190 
criteria, task requirements, 216 
evaluation, and task description, 226-227 
limits, man and machine, 234 
measures, in system evaluation, 493 
rate. matching of computer and man, 
102 

system. prediction, 480 
visual, controlled variables, 132 
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Perry, J. W., 110 

Personnel 3 ; i 
entities in selection and classification 


model, 252 
flow, in man-machine system, 247 
subsystem, 305 
supply and demand, 236 
systems, types, 237 
turnover or attrition, 235 
Personnel measurement 
average level of performance, 256 
homogeneity, 256 
objectivity, 257 
reliability, 257 
specificity, 257 
validity, 258 
variation in scores, 256 
weighting of scores, 260 
Personnel subsystem 
decision structure, 192 
flow of personnel, 247 
jobs, 240 
nature of, 237-249 
and proficiency, 242 
recruitment, 248 
reference provided by task description, 
190 
and selection procedures, 244 
system development, 18, 23 
system mission, 239 
training programs, 248 
types of personnel systems, 237 
PETERSON, R. L., 448 
PHILLIPS, J. C., 468 
Photo-interpretation, aerial, 5] 
Pilot proficiency, study of, 464 
Pilot study, in system evaluation, 506 
Pitch, auditory, 137 
Placement of men, use of tests, 428 
Planning human components, 326 
Policy formulation, in emergent situations., 
399 


Population generality, in research, 542 
PORTER, E. H., 393, 41 1 
Practice, changes in system behavior, 513 
Prediction 

and forecasting, in research, 543-544 

kinds of models, 266-269 

in research, 530 

system performance, 480 
Pressey, S. L., 350 
Prevalidation procedure, in 

measurement, 260 
Principia Mathematica, computer program, 
109 

Principle generality, in research, 542 
Problem solving 

category in task analysis, 211 

computer program, 109 

job aids, 291 

training, 222 
Procedure(s) 

emergency. training, 220 

flexibility, 173 

trainer, 354 


personnel 


Index 


training, 220 
work, and teams, 388 
work improvement, 393 
Proficiency 
air defense system, 469 
assessment, in training s 
development of measures 
goal training development. 311 
measurement, and accessibility of be- 
havior, 432 
measurement, 
ferences, 427 
measurement, definition, 420 k 3p 
measurement, related to selection, 242, 
246 
measurement, 
425 
measurement, uses, 426-430 
Navy teams, 468 
and personnel subsystem, 242 
pilot study, 464 23 
Standards, as references for system, 42 
see also Proficiency measures and Tests 
Proficiency measures 
applications, 463-470 
characteristics, 421-426 
check lists, 433 
component vs. global, 452 
contamination and bias, 440 
correlated-behavior, 460 
criterion-referenced, 421 
defining purposes, 431 
group attainment, 429, 430 
infrequent events, 434 
multiman systems, 468 
norm-referenced, 422 
on-the-job, 454 
placement, 428 
precision and relevance, 440-450 
process vs. product, 450 
quality control, 428 
rating scales, 433 
reliability, 443, 444 
sampling of objectives, 435 
simulation, 435, 457 
standardization, 452 J- 
standards of reference for system deve 
opment, 423 
system performance, 429 
training, 462 
troubleshooting, 466 
types, 453 
validity, 447 
weighting performance, 437 
see also Proficiency test 
Proficiency test 
construction, in training study, 323 
dimensions, 450 
Observer judgment in, 451 
reliability, 446 
uses, 426 
validity, 447 
see also Proficieney measures 
Programming j 
computer, automatic, 84 


ssessing individual dif- 


training and education, 
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computer, languages, 83-86 
computer, simplification, 83 
computer, steps in, 81 
computer, use of subroutines, 83 
conditions in human functioning. 45 
contingent, in computer displays, 103 
heuristic, and job aids, 289 
heuristic, computers and men, 93 
instruction, 332 
iterative and Monte Carlo methods. 83 
thinking activities, 288 
see also Programs 

Programs, computers, 80-88 
bugs in, 87 
game-playing, 109 
human learning simulation, 108 
random, 108 
writing of, 87 
see also Programming 

Psychological principles, and system de- 

sign. 160 
Psychological 282. 


Research Associates, 


Psychological scaling, 154 
Psychologist 
üctivities in system development, 16 
interest in systems, 10 
participation in development, 116. 159 
participation in system design. 160 
Psychophysics 
data in function allocation, 152 
methods, 121 
probabilistic data, 154 
relation to system design, 154 
Psychophysiology, data in system design, 
152 


Psychotechnology, in training development, 
310 


Qualitative personnel requirements infor- 
mation ( QPRI). 491 

Quality control, use of proficiency meas- 
ures, 428 

Quickening, displays. 104 


Radar 
displays, design. 176 
map, in system evaluation, 496 
returns, information in, 142 
troubleshooting, 70 
vigilance task, 117-118 
weather, as system example, 143 
Rand Corporation, system research labora- 
tory, 413 
RATH, P. J., 350 
Rating, job performance. 462 
ating scale. proficiency meas 
Readability, displays. 174 
ecall, training for short-term, 219 
Reception of information. 205 
Receptors, tactile and kinesthetic, 144 
Recoding, system task, 166 
Recruitment, system personnel, 248 
Redintegration. pattern construction, 169 
Rees, D. W., 297 


urement, 433 
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Reliability 
computers and men, 95 
controls design, 181 
personnel measures, 257 
proficiency measures, 443 
Replication, measures in 
classification, 255 
Research 
basic and applied, in training, 311 
choice and definition of problem, 526-528 
definition of units, 538-539 
engineering, forecast formula, 530 
engineering, use of schemata, 531 
"field laboratories," 31 
generat principle vs. population, 541- 
job aids, 286-298 
level of analysis, 538-539 
methods in air defense study, 26-28 
methods in system development, 26-28 
number of observations, 537 
operational definition, 533 
options, 517 
psychological, in system development, 30 
representative design, 536 
representing variability, 535 
statistical inference, 539-541 
system development, 21-25 
system evaluation, 503 
systems, approach, 488-503 
team training, 411-415 
theoretical, use of model, 529 
univariate vs. multivariate design, 536 
Research and development, training pro- 
grams, 310 
Research Group in Psychology and the 
Social Sciences, 25 
Response(s) 
adequacy, in task description, 201 
autonomous sequence, 43 
ballistic, 146 
categories, 43 
factors, in system design, 164 
flexible sequence, 43 
human, relation to stimulus, 129 
human, transfer functions, 145 
human visual-manual, 147 
input-output relationship, 43 
mechanisms, in task analysis, 214 
motor, training, 225 
sequences, in human performance. 165 
transient, in human control. 146 
unitary, 43 
Retention 
of codes, in task analysis. 210 
long-term, 208 
requirements. 208 
short-term, 208 
training for short-term, 219 
REUDER, M.. 66 
Riccianpi, F. M.. 353 
RicHLIN, M., 455 
Riesz, R. R., 136 
RITCHIE, N.. 174 
RoacH, E. G., 319, 320 
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Rods and cones 
distribution, 126 
sensitivity, 127 

ROCHESTER, N., 108 

ROETHLISBERGER, F. J., 396 

RocGE, H., 330 

ROSENBLATT, F., 107 

ROSENBLITH, W. A., 137 

ROWLAND, L. S., 130 

RupMonE, H. W., 140 

Rules, interpreting, 51 

RULON, P. J., 466 


Safety, controls design, 181 
Sample selection, 265 
Sampling of performance, 
measurement, 435-440 
SAMUEL, A. L., 110 
Saupe, J. L., 467 
Scale reading, study, 175 
Scanning, system task, 65 
Schemata, engineering research, 53] 
SCHENCK, H., Jr., 111 
SCHIPPER, L. M., 177 : 
School disaster, system training, 407 
SCHWARZ, P. A., 458 
Science : 
and engineering, purposes, 525 
theoretical and engineering, 524-526 
Score(s) 
selection and classification model, 255 
variation in personnel measures, 256 
Search and scan, in task analysis, 205 
Search task, computers and men, 93 
SEIDMAN, D., 337 
Selection 
decisions, responsi ility, 246 
equipment and facilities, 398 
human functions, 57 
instruments, choosing, 245 
manager, in emergent functions, 398 
procedures, in personnel subsystem, 244 
relation to task description, 189 
sample, in selection and classification, 
65 


proficiency 


types of instruments, 244 

see also Selection and classification 
Selection and classification 

attributes, functions, 252 

criterion attributes, 251 

criterion data, 261, 265 

criterion measures, 263 

jobs in man subsystem, 240 

matrix, 253 

measures of attributes, 254 

measuring instruments, 256 

model, 250-266 

model, entities, 252 

personnel flow, 247 

prediction, kinds of models. 266-269 

predictor attributes, 251 

predictor formulas, 270 

predictor selection, 269 

prevalidation procedure, 260 

and proficiency, 242, 246 


Index 


purposes, 232 
relation to type of personnel system, 237 
System mission, 239 
training programs, 248 
Selection and synthesis, 169 
SELFRIDGE, O. G., 94, 108 
Sense modality 
display design, 174 
engineering design, 123 
Sensing 
auditory capabilities, 134-144 
characteristics, 124-125 
computer systems, 101 
conditions, 45, 46 
gustatory sensitivity, 144 
limitations, 59 
olfactory, 144 
overload, training for, 403 : 
thresholds, and filtering conditions, 46 
touch and kinesthesis, 144 
trained Observers, 46 
vibration, 144 
visual, characteristics, 126-134 
visual acuity, 39 
Sensitivity, rods and cones, 127 
Servo mechanism, human control. 145 
Set, in human functioning, 45 
SHANNON, C. E., 110 
Shape, in system design, 134 
SHARP, L. H., 66 
Suaw, J. C., 93, 109, 110 
SHELLY, C., 164 
SHOCKACTION project, 317, 522 
SHOEMAKER, H. A., 336 
SHOWER, E. G., 137 
Shunting 
human functioning, 44 
identifying, 47 
interpreting, 50 
SIDMAN, M., 538 
SIEGEL, A. I., 455 
Signal classification, 166 
Signal-to-noise ratio, 141 
Signal variability, 172 
Significance 
levels in research, 539 
statistical and practical, 541 
SILBERMAN, H. F., 351 
Simon, H. A., 93, 109, 110, 396 
Simplification of tasks, 161 
Simulation 
air defense system, 361 
development of vehicle, 486 
entire system, 485 " 7 
environment, in system evaluation. 48 
environment, in training, 381 
fidelity, and transfer, 372 
fidelity, transfer, and cost, 374 
intellectual Processes, 108 
neural structures by computers, 107 
proficiency measurement, 435. 457 
System design, 367 
System evaluation, 482-488 
technique for studying teams, 413 
see also Simulator(s) 
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Simulator(s), 354 
crew training, 359 
flight, 355 
miniature armor battlefield, 360 
Naval electronic warfare, 360 
tank-crew proficiency, 459 
Skill(s) 
specification for training, 315 
trainer, 353 
weighting, in proficiency measurement, 
438 
SKINNER, B. F., 538 
Slides, 346 
SLoAN, L. S., 128 
SMITH, J. F., 465 
Smitu, R. G., JR., 308 
SMrrH, W. M., 175 
SoLoMoN, P., 166 
Sonar 
example of system task, 166 
HUK-Master system, 97 
Specifications, human characteristics, 234 
Specificity, in personnel measures, 257 
Spectral composition, visual requirement, 
133 
Speech 
articulation, 141-142 
distortion, 140-142 
masking by noise, 141 
SPENCE, K. W., 332 
Stability, in personnel measures, 257 
Standards, proficiency, 423 
Stark, E. A.. 323 
Statistical inference, in research, 539-541 
Statistics, in research, 541 
STEINBERG, J. C., 140, 142 
Stevens, S. S., 137, 138, 139 
Stimulus 
characteristics of job aids, 292 
control by job aids, 296 g . 
Stimulus-response, non-linear relationship, 
129 
Storage, long-term, computers and men, 93 
STOUFFER, S. A., 328, 331 


ision-making, 214 
g. 408 
Structuralism, 53 
Submarine 
attack trainer, 360 
predictor display. 106 
Subsystem (s) 
level, and task description, 195 
operational, 304, 314 
personnel, 305 
system evaluation, 492 
training, 305, 308 
types. 304 
Supervisor 
effectiveness, 462 


job factor. 240 |. . : 
ratings. in proficiency evaluation, 243 


Supply and demand, man and machine, 
23 


SVIMONOFF, C., 176 
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Swain, A. D., 153 
System (s) 
air defense, 130 
aircraft control and warning, 361 
analysis, in emergent situations, 399 
behavior changes, 513 
comparison with alternative, 479 
complex, human problems, 369 
computer, design principles, 100 
concept, implications for research, 519 
definition, man-machine, 16 
design, and simulation, 367 
development procedures, 4 
evaluation, design and control, 503-514 
evaluation, hypothesis, 499 
evaluation, levels, 491 
evaluation, monitoring techniques, 495 
evaluation, performance criteria, 493 
evaluation, purposes, 479 
evaluation, using simulation, 483 
evaluation studies, approaches, 503 
experiment, crew preparation, 511 
experiment, data-collection, 510 
experiment, design problems, 511 
experimentation, 507, 508 
goal-oriented exploration in training, 380 
goals, and human capabilities, 116 
HUK-Master, 97 
improvement, 480 
inputs, simulating, 483 
interceptor aircraft, 2 
job aids, 274-279 
learning, air traffic control, 72 
life-cycles, 14 
man-ascendant and machine-ascendant, 
304 
multiman, proficiency measurement, 468 
operational effectiveness, 36 
organization and department structure, 
196 
parent, 302-303 
performance, improvement, 480 : 
performance, proficiency measures im, 
429 
performance criteria, 22 
personnel, types, 237 
planning for design, 3 
power distribution, | 
purpose, related to research. 519 
relation to training, 303-309 
requirements, 216 
SAGE, 361 
simulation, 485 
skills and understanding, 379 
solar, 15 
stressing, in training, 381 
theory, 25 
weather radar, 143 
System design 
data on human performance in, 124-125 
relation to task description, 191-195 
System Development Corporation, 
487 
System operation 
levels of description, 195 


338, 


558 .- 


System operation (continued) 
task description, 195-202 NT 
System requirements, human capabilities, 
2 
System research, scientific approach, 488- 
503 
System task(s) ed 
accumulating and summarizing, 167 
action selection, 170 ^ 
cause and effect attribution, 169 
coding, 62 : 
critical-cause selection, 170 
decision-making, 172 
effect evaluation, 171 
human functioning, 62-72 
information processing, 171 
load-sharing, 71 
monitoring, 66 
output-processing, 168 
pattern construction, 169 
performed by computers, 93 
recoding, 166 
scanning and detection, 65 
selection and synthesis, 169 
signal detection and classification, 
time line analysis and prediction, 
tracking, 67 
troubleshooting, 69 
value weighting and destination routing, 
168 
System training 
announcing, 409 
costs, 409 
design of problems, 408 
designing, 405-411 
determining strategy, 408 
functions in which needed, 407 
knowledge of results, 382 
operational, 361 
postexercise discussion, 410 
principles, 379 
simulation, 380 
system development, 20 
team training, 344 
training team, 409 


166 
170 


Tab-Test, in troubleshooting, 467-468 
Tank Commander's Guide, 321 
Tank crew, proficiency, 469 
TANNER, W. P., 154 
Task(s), 62-72 
definition, in proficiency measurement, 
430 
discrete and continuous, 200 
heuristic programming, 93 
pattern recognition, 94 
relation to human functions, 58 
search, by computers and men, 93 
storage, long-term, 93 
structure, task analysis, 215 
varieties supported by job aids. 284 
see also System task(s) 
Task analysis 
behavioral structure, 202-215 
goal orientation and set, 204 


Index 


implications of task structure, 215 
interpretation, problem solving, 211 
motor responses, 214 
reception of information, 205 
related to controls design, 179 
retention of information, 208 
task description, 197 
Task description 
behavioral taxonomy, 193 
contingencies, 199 
detailed, 201 
environmental conditions, 199 
human error, 200 
mechanics, 198 
mission segments, 199 
objective performance criteria, 190 
performance evaluation, 226-227 
purposes, 188-191 
reference for personnel subsystem, 190 
relation to system design, 191-195 
sample, 203 
statement of functions, 198 
System design, 188 
System organization, 196 
task analysis, 197 
varieties of tasks, 200 
Task information, training design, 216-226 
Task requirements information, 216 
Taxonomy, human tasks, 193 
TAYLOR, D. W., 53 
TAYLOR, E. K., 440 
TAYLOR, F. V., 67, 104, 153, 367, 478 
Teaching devices, automated, 350 
Team(s) 
definition, 388 
effectiveness, and OJT, 405 
functioning, machine alteration, 392 
functioning, work procedures. 393 
functions, and training, 388-395 
functions, emergent and established, 395- 
399 
functions, meaning, 390 
functions, perspectives, 396 
functions, predicting, 396 
proficiency measurement, 471 
Team training 
adjustment mechanisms, 403 
emergent situations, 399, 404 
error analysis, 401 
interdependencies, 401 
meaning, 391 
Orientation to goals, 400 
programs and equipment, 358 
research, 411-415 
sensing overload, 403 
System development, 20 
system training, 344 
technology, 399.405 
TEEL, K. S., 448 
Television, closed-circuit, 346 
Temperament, man and machine, 234 


Temperature, effects on performance, 149 
Test(s), 254-260 


Army, 323 
Army general classification, 336 
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job knowledge, 428, 461 
paper-and-pencil, 46 
proficiency, in training study, 323 
proficiency, reliability, 446 
proficiency, troubleshooting. 466 
proficiency, validity, 447 
psychological, in selection, 244 
verbal, in proficiency measurement, 461 
Testing, system, 7 
Theoretical research, paradigm, 520 
"Theory 
psychological, and system development, 
30 
system, 25 
Therblig, controls design, 179 
Thinking 
category of human functioning, 56 
programming for job aids, 288 
THORNDIKE, R. L., 247, 262, 265, 441, 443, 
446 
Threshold 
auditory, 135 
detection, in vigilance task, 118 
TIEDEMANN, J. A., 360 
Time compression, computer displays, 103 
Time-line analysis, 170 
Time-sharing, task description, 200 
ToLMaN, E. C., 538 
Tracking 
analysis of responses. 146-147 
capability of human operator, 147 
HUK-Master system, 98 
operator function, 145 
system task, 67 
Trainees, aptitude, 335 
Trainer(s) 
concept, 352 
individual, 350 
procedure, 354 
skill, 353 
see also Training device(s) 
Training 
achievement in armor, 320 
aptitude requirements, 313 
automated instruction, 337 
basic and applied research, 311 
content, 225, 371 
criteria, and evaluation, 377 
curriculum, 325 ^ 
decision making and problem solving. 
222 
decreased time and costs, 313 
design, psychological considerations, 369- 
83 


design in system framework. 365 
development, goals, 311 
development of programs, 310, 314 
environment, 406 

frequency, 377 

functionally complete unit, 380 
human engineering, 367 

human functions, 57 

identification of work objects and ac- 
, tions, 218 

increased proficiency, 312 
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individual, 20 
individual or component, 345 
input to, 308 
media, varieties, 334 
methods and devices, 371 
military, development project, 317 
motivation, 328, 329 
motor response, 225 
nomenclature of work objects and ac- 
tions, 218 
objectives, determination, 315, 325 
objectives and requirements, 370 
on-the-job, 356 
operational, 19, 361 
organization, 217 
orientation, 217, 400 
output, 308 
planning program development, 365-369 
procedures, 220 
proficiency measures, 462 
program, construction, 316, 321 
program, integrating into operations, 382 
programs, and personnel subsystem, 248 
programs, research and development, 
310 
programs and equipment for team, 358 
psychotechnology, 310 
purposes, 306 
relation to job aids, 294 
relation to systems, 303-309 
relation to task description, 190 
school vs. on-the-job, 327 
search for job-relevant cues. 219 
selection, classification, 248 
short-term recall, 219 
subsystem, 305, 308 
system, 20 
team, 20, 391, 399 
team, error analysis, 401 
team, interdependencies, 401 
team, sensing overload, 403 
team and crew, 357 
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